BPM mapping tools – integrating data?

30 11 2009

As I was shifting through my usual amount of mails this morning, I spotted a google alert regarding an article titled “BPM must integrate people, documents”. Now I thought that’s a tad obvious, and promptly went to read the article thinking I had missed something….And basically I hadn’t…

A good point was raised though, that most BPM software targets business analysts and project managers. There are usually options to utilise SDKs but these are often a low priority as the main selling point is almost always “out of the box” demonstrations aimed at business analysts. However, am I the only one that sees a flaw in this?

BPM design tools. Are they flawed?

Now putting together demonstrations and targeting analysts are all good things to a degree, however in the world of BPM I still maintain that too much is done to show a great demo, allow analysts to “model” processes and not enough is done to provide real integration and back end flexibility. What do I mean by this? Well, systems integration in the “real world” often require a lot more work than simply setting up some connectors with common integration points. Often you need to call functions from other systems, pull in data from more than just a single source, carry out some complex processing, update multiple systems etc all within a single step. Can you really do all this from a fancy modelling tool….My guess is no.

Modelling tools give power to the business

This is so very true. Business can make tweaks to their processes and make them happen immediately, which is great. Many BPM solutions almost live and die by their modelling tools / facilities and spend vast amounts of time, effort and money in making these better and more flexibile. There are so many benefits which many other articles will and have talked about. But again, in my experience, many larger systems aren’t just “out of the box”, nor are many smaller systems. They have applications written in .NET for example, that use the process map for guidelines and routing business rules. The application itself then contains its own “business layer” (if written well) which contains certain inbuilt business rules – typically regarding calculations and integration with other LOBs.

I have worked on numerous BPM projects that work in this way, providing lots of applications for different steps within the process map, using the process map for merely setting statuses and routing information. The application itself holds all the business logic and integration code. Even on the more complex solutions, an application was delivered in place of any out of the box front end components. Business rules were built into the application to interpret the process map, which in theory did give back more power to the process map, regarding statuses, routing, cooling off periods etc.

In this case you can see that the modelling tool gives power to the business on when and where to route work for example. Even such things as options the user can select, cooling down processes, awaiting other tasks etc. However, the real integration and bringing people / data together, happens in the business application, written by developers specifically for that client requirement.

But what happens if the business analyst wants to make a change to the process that involves carrying out additional calculations at a particular step or pulling in more content / data from other platforms. He can make the changes as he sees fit in the map, but what about the actual rules etc? More often than not a “change request” is raised and someone in the development team has to make a modification to the code of the application, which then obviously needs to be tested in a model office environment before being released. So even though the business feels like it has ownership of the process map, it doesn’t really have great ownership over how the map actually works as a process for the business. In this scenario the actual time taken to make the change is quite lengthy. Let’s break it down:

  1. Business identifies the need for change
  2. Business analyst is involved and investigates the options
  3. Business analyst provides a new solution
  4. Business analyst updates / creates a new business process map
  5. Change request is made and passed on to the software developer
  6. Business analyst updates the process map
  7. Software developer makes the required changes
  8. Process map is then tested (UNIT test)
  9. Software is then tested against the new map (UNIT test)
  10. Software is then sent back to a testing environment
  11. Testing environment tests the new map and all the related applications / scenarios
  12. New process map and application is released to the live environment

As you can see that’s quite a list of things that must happen.

What’s the answer?

It’s simple. Let’s let people do the jobs they are supposed to do, and let’s let software provide us with what it is designed to do, nothing more and nothing less.

So with this in mind. imagine a business analyst who models the business process. He / she put together a complete map with users, groups, security, calculations identified, function calls to other LOBs etc. They then model this in Visio for example with some accompanying documentation.

Once this is all complete, let’s move this over to the actual BPM software, which should allow all these things to be included into the “map”, even execution of complex code, calculations and integration etc. Please note, the BPM engine shouldn’t be confused with the BPM front end at this stage. The user will never see the “engine” as such. So our BPM specialist (which is more than likely at this level to be a developer), makes the changes to the BPM map, codes the required calculations and integrations and completes the map. This is then “tested” before being published. Please note, no application code has changed, there is no need to test the applications etc, rather just the map itself.

The last part of our solution is our front end. This should be written to only present data to the user and allow them to interact with that data, people or other systems depending on the BPM map (which holds all the rules / calculations etc). Because map changes are made in the map and are completely separate from other deliverable code, namely our actual application / front end, the whole process of making a change to the process map and having it made live is far shorter, less likely to fail, far more flexible and in the end, much cheaper.

Let’s review:

  1. Business identifies the need for change
  2. Business analyst is involved and investigates the options
  3. Business analyst provides a new solution
  4. Software developer updates / provides a new map
  5. Process map is then tested (UNIT)
  6. Process map is sent to testing environment
  7. Process map is made live

We could even add in: 8. Full testing of the process map and application; and we are still 4 points shorter in our list of things to do. In addition, each point in this second implementation is not as time consuming as those in the previous list…

The downside

Because this type of BPM system / solution doesn’t display a great map for business analysts with lots of feature rich BA tools, it doesn’t really quickly and easily demonstrate BPM. A system like this won’t allow a BA to make a change to the process map and have it published immediately. All of this means that this kind of system, though in many ways far superior, may be viewed in a demonstration as a simpler tool with fewer capabilities.


When choosing your BPM platform, really focus on your actual requirements. Have more than just a rough idea of your processes and desired outcomes. If you have an in-house IT department, make sure you bring along a senior member of that team to any demonstrations / meetings.

Remember, a demonstration is not the same as a solution and don’t fall for the statement, “we have an SDK you can use to integrate”. This almost always means you need to build those more “complex rules and calculations” into your own application / step processor, not in the process itself.

BPM mapping tools are great; they deliver a powerful and rich environment that is fantastic for demonstrations. But make sure the BPM map itself has the capabilities and flexibility to meet your requirements without the need for you to build essentially what should be process rules into your own step processor / front end application…

Even with BPM there is still a process that you must follow to get your BPM system to work for your needs…



12 responses

30 11 2009
Tweets that mention BPM mapping tools – integrating data? « Andrew One Degree’s Blog -- Topsy.com

[…] This post was mentioned on Twitter by Andrew Smith and Andrew Smith, Andrew Smith. Andrew Smith said: @JohnBJansen Your thoughts on this would be really valuable http://tinyurl.com/ygbzs3f […]

30 11 2009
Sanooj Kutty

Very apt postm Although I felt that its more comprehensible for IT savvy and experienced readers. Quite simply, the post guides you through two key aspects. 1. Do you want to model and manage your processes? Or
2. Do you want to automate and manage your processes?

Once you are able to answer these, read this enlightening post and see the true light it spreads.

1 12 2009
Sanooj Kutty

I have taken time to read this again. The two change process options given by you have their own set of benefits, I feel the clear difference here is the role of the Business Analyst, expecially, in the second case, where the Business Analyst only provides the solution and update of the Process Map is executed by the Developer. This definitely speeds up the change management process.

The big Q here is not in just mapping the process, but, one also needs to analyse the scope of the activities within the process. If the activities requires automated functionalities with multiple LOB systems, it might be not as smooth as mentioned above.

While the recommended approach is surely backed by me, I would provide a clear warning to evaluate the scope and impacts and modify it where necessary.

1 12 2009
Andrew Smith @onedegree

I hear you with regards to evaluating the scope and impacts. Another reason why I prefer this type of solution is that many organisations dont actually know what they want / expect or can do at the time of purchase. Typically scope usually creeps along, even after a solution has gone live. This type of solution provides the maximum amount of flexibility and integration capabilities. In addition it means the BPM platform and processes really can grow / scale with the organisation and its process requirements, without the need for expensive upgrades / platform change or yet further investment…

4 12 2009
John Jansen

I agree on the conversation so far. Would bring in an extra element. With some of the BPM tools it is possible to prototype. Then you translate the business process model into a application as a prototype. This helps enormously the communication between business analyst and ict people, but also it is a very good way to communicatie with the business the effects of a give process model. As we all know it is difficult for business people to “translate” a model into “the real world”. Prototyping helps a lot. Why are most BPM tools not developed in such a way?

6 12 2009

By now everyone is sold on the fact that it is Business that should drive BPM and so most influence in decision is likely to come from business teams. This is why we find most BPM tool vendors are directing their pitches to this audience – and as you mention, snazzy demos and presentations seem to sell the idea of ‘its easy, you can do most of it yourself’

It is important to keep in mind that BPM is not a way to marginalize the role of IT, but rather it merely helps relieve IT of the business aspect of technology and puts that part of the job back to business. That way both teams focus on their core strengths and the benefits are more emphatic.

This is especially true for integration centric implementations.

This is a great discussion and a very relevant one!

7 12 2009
Sanooj Kutty

Yes, Business definitely drives BPM. Its the deployment of a process in an IT environment that brings IT into the picture. For eg., if the process has an activity to “Send by Courier”, you would see non-IT tasks like packaging and physically sending the package. A BPM tool/modeller must be able to capture all relevant aspects of a Business Process helping identify digital and automated activities to plan out integration when automating the Business Process.

Also, if your Business Units include non-IT leadership or users, you may need to handhold them when defining the process. I believe a BPM (especially integration centric) roll out has 2 simple phases – Definition and Implementation.

Definition – Led by Business, Supported by IT
Implementation – Led by IT, Supported by Business.

The basics of Business applies here – Your core competence is important but not at the expense of the sub-competencies. This is why the conventional SDLC may not be really advisable here. IT must be present in the complete Business Analysis phase and Business must be present in the full Solution design stage. No matter how well your documentation is, its never the 100% picture. And in most cases it is a Third Party that is contracted to do your Business Analysis, Design and Development

11 12 2009
Andrew Smith @onedegree

Interesting that BPM sees integration with other applications / companies etc as key across complex networks. The big challenge is the way BPM is implemented, with complex networks and integration not really being handled becuase of the use of mapping tools to deliver both the “map of a process” and the “implementation” of that process…. The following is well worth a read…


8 01 2010
Incorporating automation into your processes « Andrew One Degree’s Blog

[…] can be achieved with regards to integration. This is something I have blogged about in the past, https://andrewonedegree.wordpress.com/2009/11/30/bpm-mapping-tools-integrating-data/ with many automated “steps” you will require the services of a developer, hopefully your chosen […]

14 01 2010
18 01 2010
Max J. Pucher

Andrew, you are pointing at the right problems. Good!

I am just not at all capable of seeing that the solution you discuss improves much. The reality of BPM usually means that creating and changing processes is so complex that even the 12 step process is not enough. Typically you need to even go into integration testing with the backend and with all the other processes, especially if you are trying to reuse process components and whole processes as much as possible. If you use the same process for 5 departments, it now means that a change that one department needs has to be verified by all five and in the end tested by all five. What a MESS!!!!

The problem is that the concept of BPM is flawed from the outset. It treats the business as an animal with a clever head and a dumb body, when it is all people that act intelligently. If you want to truly give the business ownership of the process, they must be able to create the process themselves on the fly without flowcharting and business analysts and testing. The process is executed as needed and not as designed. I do not believe there are these processes that can be automated and if yes, then they should not involve people at all and they become appiications.

What is the best proof that I am seeing this more realistic than BPM proponents? The recent sale of Human-BPM vendors Savvion and Lombardi. They clearly saw their market place shrinking. Strange, when they should have been the ones to grow faster than anyone else with their people empowering dynamics. The problem is that they were still flowcharting and just selling pre-coded processes through large partner networks.

18 01 2010
Max J. Pucher

Let me add the following to my previous post:

Yes, there is no need for flowcharts or maps for actual processing. There is a benefit to have maps for understanding but they ought to be created not in advance in Visio but from the actual execution by the users.

On top of that the people who OWN the process know typically the best how to make it work if they are given the right authority. Outsiders will always be outsiders and that inlcludes, analysts, consultants, IT departments and process centers of excellence.

Finally, what you describe as a base system seems to be fairly similar to what we provide with the Papyrus Platform. To make all this work you need integration functionality, user definable GUI, user definabel business rules, a powerful prohect management and deployment mechanism.

So in the end I AM NOT DISAGREEING with you. I feel that you did not go far enough in your analysis of the real world problems that process management if facing.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: