Integration Development is the connection of two or more systems to share data, information (note that is different to data) or pass control for a flow. A basic integration can be the uploading of a local spreadsheet (perhaps containing client data or a new order) into a system for storage or further processing. Every business however small has a litany of systems often covering a wide range of business processes from various software vendors, software development company’s or in-house solutions (that excel spreadsheet that seems to control the procurement). The simple solution for a lot of businesses is to simply give the task of managing data flow / control between the systems to an administrator. The task of exporting, validating and re-entering the data can often be tedious but a critical part of any business operation. The purpose of this blog is to look at how we can understand the current integrations in use, how we might optimise and a quick discussion around a single piece of software to do everything (ERP for example) or best of breed software applications that can be integrated.
‘We don’t DO integration’
A lot of our clients claim they don’t have any systems that require integration, or they don’t have any formal integrations. If we remember that a human action of moving data between systems is also an interface – then often the answer changes to ‘yes we do have a lot of data being moved between systems’. Often the finance system is the system with the majority of integrations – as the system that provides reporting and management control (often our clients manage their businesses based on the financial numbers alone) it is understandable. How many orders are entered here after being confirmed, how many payments are entered, bank statements reconciled, expenses loaded, payroll – you get the idea.
The start to understanding where you store your data, what integrations you have and whether you can simplify or streamline is to follow these simple steps (a simple excel sheet with columns will help here):
- Document each of your systems that you use (that Excel spreadsheet should be included here and also your website)
- For each system document what data is stored
- For each of the data types per system indicate which is the master and which is a slave (there can only be 1 master across your organisation – typically this is where you go to get trusted data/reporting)
- For the systems identified as a slave indicate which other systems the data is shared with (ie that Excel sheet is manually typed into System A etc)
- For each of the interactions in 4 ensure you detail who is involved/what is involved in the data flow
At this stage you should have an excel sheet showing what data is stored in which systems and what the data connectivity / flow between the systems is. If you are feeling brave then now is a good time to use Paint/PowerPoint or Visio and for each system show the interconnection (from a data perspective) between the systems (an entity relationship diagram or known by many other names) – this will help you visualise the connectivity between systems.
OK we have lots of systems storing lots of data
Next is to look at the data being stored across different systems and identify opportunities to optimise.
Data Duplication – are you entering the same data twice – that’s a simple optimisation (can you export the data from one to another, do you truly need the data in both systems) for example. When developing software, we often ensure that we capture data once, we allocate that to one master system and we manage the flow of data between systems.
Use of features – are you using the software / solution to its best effect – perhaps you have an updated version, or there are plugins or add-ons that you can purchase/use which will allow you to more easily enter or transfer the data. Ask the software vendor, try a quick internet search and be open to changing your processes or procedure slightly to accept the standard.
Data Normalisation – are you capturing or transferring data that simply isn’t used by that system? Remove any data flows or data being loaded into a system that isn’t required – it might sound obvious but with data protection regulations ensuring you are only keeping data in a minimal number of locations will help with any data audit that you might choose to perform. Don’t be tempted to store everything everywhere just in case – this complicates updating, removing or creating data (adding cost and risk of incorrect data being used).
Will it really save me money?
OK so depending on the costs to change the process / purchase costs of any add-ons to integrate (or that mobile app to save time on data entry), then this is a sensible question – however when looking at costs we should assess how much human effort is involved in the synchronisation, how much that costs and on what basis (or for software how much the maintenance cost is, licensing costs etc). We have helped clients to move administrators into supporting value add roles such as sales or marketing and away from data management or covering the cracks between their systems – with solid business cases that don’t always cause job losses). Of course this will vary from case to case so you should develop a business case to justify any costs or changes prior to blindly accepting things will be better – consider the implications on reduced data storage costs, reduce effort in auditing/harmonisation efforts later and reduced risk in the data being accessed by somebody who shouldn’t have access (your data security policy should ensure you know exactly where your data is at all times and who has access and has accessed).
So why not have 1 system doing everything
Whilst it might be tempting to have one system managing and controlling the whole business this isn’t always feasible or practical – rather have a solution landscape that supports a common integration framework or protocol. Looking for the best of breed solution that delivers business benefit based on your budget and ensuring that can easily be integrated to those systems that require the data can often deliver a better overall business system, at a lower cost and reduce any risks of a single system failure. Whilst there is additional cost and overhead in managing the integrations and data flow – these are often traded against the benefits the solution offers to the business. When examining the case for our clients we often look at the needs of the business (requirements), map those to the available solutions, factor in integration models and then weigh up the pros and cons of having the solution delivered by various components (some of which might be on the same system and some on different systems).
As a software development company we support integration to existing and new systems by either providing an integration hub or provision of open standard integration mechanisms to ensure any data we obtain or provide is easily accessible. This provides our clients with the certainty that should any software component in their landscape change for any reason the business process / business logic is isolated from the technical integration layer (making the change technical in nature and reducing re-test effort significantly). An example of our integrated solutions include ensuring a 24* 7 bottling line was connected to the ordering system, label printers, scanners and the warehouse management system (ensuring products were bottled and packed to order, then stored ready for loading and delivery – keeping manufacturing details printed on each label) – this required a large number of systems to be integrated and any failure meaning wasted product or a stopped line.