Adaptive BPM…No Mapping tools…

I have had many a conversation with Max J. Pucher with regards to processes, definitions, traditional BPM, maps, UI’s etc etc. Many points on which we have agreed, and many we haven’t. However, discussion is good for the soul but also in expanding your thinking – nothing is better than bouncing ideas off of another person, and I have found that some of Max’s comments and blog posts (you can read his personal blog here http://isismjpucher.wordpress.com/) do challenge my own way of thinking…

One of the key areas and things he talks about is “adaptive processes” and how processes can be spawn without a traditional BPM map, adapting based on user actions or requirements (when authorised). This effectively allows a new variant of processing to be created at a user level, based on their requirement to process that particular piece of work, there and then…. Now at first I didn’t really follow the points being raised that well, in addition, I couldn’t really grasp the real benefits, all I could see are the negatives. However, this has got me thinking more and more, and how potentially this could work with our own workFile ECM BPM implementation…

Intelligent Maps…

I am a strong believer in intelligent maps, allowing developers to define the actual engine of the process and therefore giving them all the tools they need to integrate the solution with other LOB applications, making the BPM solution ultimately more powerful and useful to the end user. I am not going to change my thoughts on this. However, with intelligent maps I propose that the “map” is the driver behind the process, yes, but this is more of an “engine” rather than the complete car, so to speak…Many BPM products place far too much emphasis on the process map and the business analyst who is trying to define the actual process, for me this is far too restrictive and reduces the intelligence within the actual processing engine (or map) and its capabilities to integrate and automate…

However, Max comments and blog posts have got me thinking, can an intelligent map be adaptive at run time? Can an intelligent map allow a user to spawn new variants or sub-processes?

Adaptive intelligent maps…

Let’s work on some assumptions (I know these are great, but we need a point to start). Let’s presume that 40% of a process has been mapped out in full by a BA. This has been translated by our developer on our BPM platform as an intelligent map, and our BPM solution is running fine. Lets also presume that allocation to “unstructured” case management makes up around another 20% of our requirement. This leaves 40% of a particular group of processes that have slipped through our system to an extent (they could always be assigned to our unstructured case management). As Max explains, in his posts, an adaptive process can cater for the rest, effectively allowing these new processes to emerge from the designed or identified process (by our BA).

When you think of an adaptive map / process in this context, you can see there is real potential and benefit to this “adaptive” thinking…Though I don’t subscribe to everything “adaptive”, I do find this idea of “emerging” processes very compelling…

Where does it fit? Lean? Adaptive? Traditional BPM?

So where does “adaptive” processes fit. The answer, I am not sure. However, it is obvious that adaptive has a lot to offer and bring to the BPM market, more so than Lean for example. With this in mind, only this week I started to get our technical director looking at how we can make our own BPM and intelligent maps more adaptive, responsive and able to provide the capabilities for new processes to emerge….Though this isn’t the complete “adaptive” picture as Max paints, I feel it is a step in the right direction and one that brings benefits from the “adaptive” and “traditional” BPM corners…In addition, can ECM be more adaptive? The answer, YES….

Do people get BPM and Case Management? For some, Case Management is critical…

There has been yet another social case in the UK where, it seems, case management just simply is poor. I am not talking the technology here or anything like that, just the ability to manage cases in many cases (far too many in the public sector – though the private sector can be equally poor) where they simply cannot manage cases…

This has become a bit of a discussion thread on Twitter, and a great question was raised by @tomshpepherd, “What steps do we as a whole take to help them get it?”. In this case, them was aimed at the public sector, however, it can be aimed at anyone who hasn’t invested in BPM / workflow or case management…

So what steps can we take…Well here are a couple of things I believe the industry has to do in general.

Keep things simple…

With BPM, Case Management and workflow, the problem is that the jargon and marketing surrounding these technologies can turn people off of them. My experience of many decision makers (more so in the public sector) is that they want the basic facts, and they want them without the frilly marketing bump. Unfortunately just wonder around the web looking at topics surrounding case management (more so BPM and workflow) and you will find it things aren’t in simple terms.

By putting things in simple terms I feel that the benefits of these technologies are more clear and understood. For example, if you said to a decision maker that Case Management provides:

1. You won’t lose sight of cases. Things are always visible
2. You can track who is working on a case and when
3. You ensure all required tasks are completed for each case
4. You can process more cases easily

I am sure they will be interested in what you have to say. However, often we don’t say this. Rather we talk of audits, accountability, process efficiency, reduced cycle times, SLA’s, process maps and possibly “lean” (in the case of BPM) etc. If you don’t really grasp BPM or Case Management, then this really isn’t going to help…

Provide relevant demonstrations

Far too often I have seen demonstrations focus around the good old “Expenses” workflow – for Case Management, BPM and workFlow solutions. While the expenses case is good, it really makes things hard to understand how Case Management or BPM fits into your own requirements, especially if your requirements are more personal, such as Social Service Child care for example.

Relevant demonstrations make it easier for people to grasp the real benefits of the technology and your platform. If you do this well, they will soon be telling you how they can use your system…

Integration, Integration, Integration…

This is so so so important, yet it is often treated as a “simple” thing, almost dismissed by sales and marketing as “yeap, ofcourse we can integrate” – without providing real information. Integration is key to these types of systems working well. A system that integrates with other systems, departments, divisions etc brings far greater value and makes the agents / workers life far easier (which means they do a better job). When talking about Case Management and BPM – we should all take integration very seriously and go in-depth to how we integrate successfully and in a fashion that works best for the client (not just how we think it is easy to integrate).

Show real world benefits

If you can show a real world benefit, be it human based, money based or whatever – you are going to help people understand Case Management and BPM. By the way, when I say “real world benefits” I also mean real to the customer (whatever their work is)…

 The more people that understand the benefits of these types of systems, the more companies will invest in them and reap the benefits of them. This in terms of Social Workers and Child care (for example) could make the difference between life and death….Now that’s a strong statement, but it can be true…In many cases, poor case management has dreadful consequences for actual people…

Great website design or just great images and video?

This week I have had a couple of conversations on Twitter with regards to creating great websites quickly and easily. For many, templates with the use of widgets (as available through wordPress for example) is almost perfect, allowing websites to be put together very quickly. However, this only works when your web requirements are for simple use of media and presenting this to visitors of your website…In addition, people often believe that they have created “stunning” websites, when in reality, the website design is a standard format (nothing special) but consists of some great images and use of various media…This doesn’t really constitute great website design…

With a branch of my own company specialising in website design, I am often interested in looking at great new websites – especially those which people term has having great design. However, in most cases I look at the design and how the website works and fine that there is nothing new there, nothing great at all, rather the website consists of great images and use of video.

Beware templates

Template websites are great for quick website design and delivering something quickly. If you are a small business owner, and don’t require much in terms of functionality (or anything out of the ordinary) – then templates are a great option. They are quick to implement, flexible to an extent and cheap….However, be aware, that the design (even once tweaked) can look disjointed at times and not “fit” well…I have often found that “CMS” based solutions (especially open source versions) deliver websites that are heavy on template use. Modifications are made by using “widgets” and moving them around, however the actual design never “flows” that great. Another downside is that more often than not, you will be able to find another website out there using the same template looking almost identical to your own – this is never good especially if the other company is a competitor.

However, any template can be improved and look a lot better with the use of great images….

Great media = great design

This isn’t true I know. However, for the vast majority of standard website users (and even marketing people, IT people, business decision makers etc) the first thing we notice, and then think of as the website design, is the video, is the colours used, is the images and how they are shown. This means then to have a great looking website, the most important aspect is not design, but the images and video you use….

Just as it is true that great images can make a standard designed website look great, it is true that poor images and media quality can make a great design look poor.

I would like at this point to set a challenge. At the websites you visit, look at them carefully. Work out what you like, what you dislike and try to separate what is “design” from what is a great image / video. Once you can do this, you will find that websites are often very similar in their underlying design. In many cases you will also find that the “design” element is often the secondary part of the overall feel and experience you get from the website…

Your type of website

Remember that the type of website you require will also determine the type of design that can be put together. For example, a content managed eCommerce website design will be very different to something that is showing off say, a car design. You also have to take into consideration such things as readability, accessibility and SEO (many of these things are, let’s say, forgotten by a number of great designs out there). If you are in the business of selling a product or service on line – then no doubt you will want something very plain, easy to navigate and something that enables Google and Bing to easily index your content. This means instantly that the technologies you use, and what you can and can’t do from a “design” perspective is very different to our “car” website – which may not need to take into consideration any of these factors…

What are the best examples?

 The best examples of great website design seamlessly blend great use of images and video into the way the website reacts / feels. It is hard to explain – but a great design builds on the use of great images and videos and incorporates them into the whole experience. When this is done well, a website really stands out from the crowd (and you can see that – just like anything in life – you get what you pay for).

Unfortunately – far too many organisations approach web designs and say “build me a great stunning website” to which the design starts to do. However, if the organisation hasn’t got great images, great videos or use of various media the website design process is highly limited from the outset. Even in an eCommerce world, if the images of the products are poor, then the whole website design feels as if it suffers….

Remember, websites are part of marketing to an extent, so you need to ensure your material, including images, video and copy are top notch. Only then can a web designer show their worth and build some stunning web experiences….

Some examples

Here are a couple of examples I have seen in the past couple of days that I love….See how they blend great images, video with seamless design and feel of the website. See how the navigation to content interacts with the images and video etc to give a unique experience….(Forget SEO and any other restrictions here…I just like the design)

http://eu.wrangler.com/

http://www.bountybev.com/home.html

http://www.walktowashington.org/

http://www.gm.ca/gm/english/vehicles/buick/behind-the-beauty/feature?adv=94437#/HOME

http://204beech.com/category/background/

Are Stored Procedures Good or Bad? And when to use them?

There are a hell of a lot of articles / posts on “Stored Procedures are bad” or “Stored Procedures are good” all of which are written from one aspect or point of view. They never really offer much in terms of when or when not to use them for a developer (or someone at Uni or even for an application / system architect re-thinking things).  You can almost always tell the area of IT in which these people specialise in (or have their most experience) just based on their views on this subject….So what is the big problem? Why the discussion…..

Well, people like to have “golden rules” or “answers”. In this case, there isn’t a particular view that is right, nor wrong, rather only the correct view for a particular organisations requirement. Don’t leave the decision in a data architects hands, nor an application developers (both bring different skills and views to the table). So when embarking on a new project, or a database overhaul, you really need to bring together a number of key personell to ensure you use your database – and stored procedures wisely….There is no golden rule of “Stored procedures are evil” or “Stored Procedures are great”….

Understanding the real pro’s and con’s

Before you can really determine when best to use a stored procedure, you have to think about what are the benefits of using them and the negatives. Ofcourse – this is where troubles arise – some people have arguments that are valid – other ones not so valid. So lets break these down….

Pro’s

Well first off, lets think about learning this rather than going based on “what we have learnt in the past”. So, we can all agree that modularisation and de-coupling is a good thing. So de-coupling our data from our applications can only be a good thing. Great. Now how do we extract that data in the most modular fashion and having minimal reliance on anything else. The answer, a Stored Procedure. The stored procedure provides data architects (and other database roles) with the ability to maintain the database as a seperate entity (as much as can be possible). This provides great flexibility along with other benefits.

Now some will argue this could be done in a DAL (Data Access Layer), which is true, however this means that we have in effect “coupled” our database to another external layer. Now who will maintain this DAL? How accessible and easy is it to update especially if a number of LOBs are dependent on it? In addition, a DAL will be written in what language? Languages change far more frequent than our actual data – many companies will have data that is a hell of a lot older (maybe 40-50 years older) than reletively new languages. To illustrate my point, if you used a DAL written in COM+, how easy would it be for an organisation to find someone to write a web page that utilises this COM+ DAL layer in todays market place? Not great. However, can I find someone who can code that web page and have it interact with a stored procedure – oh, yes I can very easily…

Another pro – is that you can “tune” your stored procedure without really having an effect on calling applications. No code needs to be re-compiled, deployment made etc etc. All that is required is a competent data architect or administrator. This ability to “tune” is because the database doesnt have the overhead of being linked to a particular layer (or GUI if bound).

Performance is sometimes mentioned here, especially as stored procedures are compiled in some cases. Now, in recent years dynamic SQL performance has caught up – however, lets look at the bigger picture of performance. I have worked on a number of projects where a DAL layer was used, and yes, it executed the query almost at the same speed as our SP (most occasions the SP was still faster), however, for more and more complex queries and business rules, the DAL layer became slower. Why? Basically because it used data in-efficiently – bringing some data back, passing it to a business layer to implement rules before making another call to the DB. In such a case – this “technically” is the better solution – with business logic sitting outside the database, however, for performance, placing it inside the database provided far greater performance….

Finally, who actually writes the stored procedures? Often it is someone with extensive experience in this area, someone who can write the SP quite quickly and accurately. Ask yourself who will write it in a business application or DAL? More often than not the junior developer , (that could be a little harsh) or people who dont have as much experience of SQL as they could have – or experienced people who just dont have that much extensive knowledge of the data model / its architecture….In all of these cases – this can lead to poor performance queries and ultimately poor application performance….

So the supposed con’s

Many feel that you have a “vendor lock-in” which on the web – everyone hates (it seems on the web most people want the earth from their software for free and for it to be maintained by the most ethical people in the world who do it to help developers etc and not for a penny..hmmmm). In addition, how often does a company actually migrate to a different database, not often. In addition – if you write your stored procedures (sticking as closely as you can to more standard SQL) then migration may not be such a big issue. To be honest, if you migrate data – if you have a DAL or application that tie to a particular database, good luck re-coding and testing your dynamic SQL there…

Algorithm tuning is also sometimes seen as a negative, stating that millions of people (developers) tune SQL and only a handful can do that for stored procedures. This is really misguided. Developers who write SQL (even in their millions) will not bring to the table anyting more in terms of tuning than a handful of good database architects…

Security…Oh well, this is a classic issue. Many think because it runs in compiled code it is more secure. True to an extent. But how many people (developers) has access to this? In addition, database security is not as bad as it once was, with complex user/role security in place you can really lock down a database and it will only be accessed (admin rights etc) by far fewer people – with less knowing the full database schema. This last point, is probably a negative one – especially if your organisation looses a few good DB men…but its key in such cases to ensure you get proper handovers completed (be responsible as an organisation).

A big, and valid negative, is that you have far greater “power” with dynamic SQL and in a proper language. You can perform greater calculations, applie business rules etc etc. SQL is a little, well restrictive. But thats a good thing – see the next section…

Stored procedures can be highly restrictive – especially when you have a very dynamic database schema. By this I mean one which may well update itself or an application updates it due to business requirements. These are more common than you might think – though by no means are they the norm. In such cases, stored procedures and SQL are far too rigid and lacking in functionality. In such cases you really will need your own DAL…

So when to use a stored procedure and when not to…

As you may have guessed, I am more in favour of the Stored Procedure than not. However, there are those occasions when they just arent a great idea…Its identifiy when to use them, how to use them, and when not to use them for your applications that is key to wheather they are good, or bad for you…

So when / how to use them:

• Use them for basic and moderate complex typical functions (insert, delete, update etc)
• Keep your SPs as standard as possible and not over complex
• Dont be scared to have stored procedures that do contain “business logic” (do not get this confused with application logic – they are different)
• Use SPs, triggers etc to enforce data integrity (your data will last longer than your applications and their chosen language)

When not to use them:

• When your DB schema is more “dynamic” and therefore requires a more powerful language with features to understand how to use it (insert, delete, update)
• When your application requires a lot of application logic from the extracted data
• When you need to utilise a greater / more unique security model on the data (though in this case you may well use a mixture of dynamic SQL and stored procedures to get the job done)

With all of this in mind, to answer are Stored Procedures good or bad? They are great – when you use them correctly…..They are poor when your schema is too dynamic or complex and they are evil when you use them incorrectly….

In browser ECM / over the web ECM

I have been asked to talk a little about browser based ECM solutions, or environments and I thought, why not…First off, browser based ECM interfaces haven’t always been a great hit. In the early days of the web, web based applications were rather clunky, requiring lots of moving around pages to get simple tasks completed. I am not going to talk about the short comings of the web for applications as that is well documented, but, for ECM this environment proved that many web based solutions were slow, hard to utilise and, well, very clunky…

Why are ECM functions hard on the web?

Well the basic functions aren’t that hard these days. Since we have all moved along with how to use the web and our expectations of the web, so have web based ECM solutions – they have improved drastically. However, the problem is that ECM encompasses so much, not just document management facilities, rather the complete enterprise worth of content, in all its forms. Add into this the possibility of Social Media based content and of course Business Process Management (or workflow)  and you can see how this gets more and more complex. I haven’t even touched on extensibility yet either….

So why are these things harder on the web, well they are because of the restrictions the web places on applications. The biggest restriction is the web browser itself, and follow this up with security requirements and you can see why the web becomes almost suffocating for very free content based applications…

The benefits of browser based ECM

Simple, almost no installation on the client machine and the ECM platform can be accessed by any machine with an Internet connection. This means administering the system is a lot simpler and can be moved outside your normal server based type implementations. In theory, if architected well, you will also save on user licenses as the web is “stateless”, meaning you should not have to hold a user license when you aren’t actually interacting with your ECM repository.

However, don’t think you cant utilise thin client type implementations and have your UI in the web browser. You can move web based applications out of the browser with technologies such as Silverlight. This means you get the benefits of the web, without all the restrictions (especially if you choose to run in a “trusted” mode).

 

Good solutions…

If you have and ECM platform that is rather old in its underlying technology (I can’t think of that many that aren’t) you will probably find that their web based solutions are a bit of a “hack together”. The main reason behind this is that technology, programming methodologies etc have changed greatly in the past 25 years, along with user expectations. This doesn’t mean these solutions are bad, rather it means beware that they may limit you in some way compared to newer platforms…

So what good solutions are there that run utilising the web? Well I am not going to list any or do anything like that, rather I am going to suggest that when looking at ECM solutions you think / investigate the following points.

1. Technology used to deliver the interface into the web browser
2. Do you have to run your web application in the browser?
3. Out of the box capabilities / configuration
4. Extensibility of the out of the box type interfaces
5. Distributed processing
6. Integration capabilities
7. Administration

 

There are more, but I want to keep this post from becoming some kind of white paper…

What you will find is that when you get down to these questions – you will find there are still limitations for many of the ECM players when implementing over the web.

 Administration

Many web based solutions are just that, web based. However, administration and the real complexities of ECM are still delivered primarily through a traditional application (which may be installed on the server). To be honest, if you are a web based ECM provider, all features including administration should be capable through over the internet…

Distributed Processing Power

Remember the point of a web based application is that many people can connect to it, it’s available to all that need it. However, some solutions place limitations on the number of users connecting via a web server, why? In addition, some are highly restrictive with regards to what components are installed where, again why? What you are looking for is real capability to share processing power for the system. This can be in the form of P2P (a valued contributor to my posts strongly recommends this – Max J. Pucher), or distributed service architecture (my own preference). Both these methods provide vast scalability and performance and these are key when you think about the web and implementing solutions over the web / intranet…

Application Configuration

Many web based solutions provide a single look and feel and don’t allow much application based configuration. Because of this, developers traditionally built their own interfaces based on customer requirements and delivered these, making the interfaces cleaner, more relevant and incorporating such business requirements as field validation (this is always more evident when looking at web based solutions).  However, this isn’t what I am driving at. Ideally, you need the user to be able to configure parts of their user interface. This could be query forms for an example, or where menu options are displayed etc. The point is, once the user has the flexibility to configure parts of the UI, then their productivity will be increased. This is a key point, especially when we talk about my next point, extensibility.

Extensibility

This is a big big thing. Traditional ECM applications (including those not on the web) provide extensibility through their API, allowing developers to deliver applications that integrate with other LOBs, add business rules etc to the customer’s requirements (within a new application for the customer, not the “out of the box” product). This is a minimum when thinking ECM.

However, the real requirement is that the “out of the box” product, allows business rules and applications to be plugged directly into it. This is so important for ECM based solutions, as ECM within your organisation will grow and include more and more areas of what is termed content. In addition, why not allow the customer to add their own modules in there, or VARs for example, extending the way in which the application and ECM is used…

Plug for workFile Vision

Using the web for ECM is a bit of a passion of mine now, and it is one of the key driving points behind our own ECM platform (workFile ECM – http://www.workFileECM.com ). When working with many other ECM players (as a consultant) I did notice short comings and wanted to get my own platform together that was designed for the web, pure and simple…workFile ECM is a baby, and already we are improving how it works over the web… One of the restrictions to the administration of workFile ECM based applications was the web browser itself, with our own modeller / administration application working in a browser – but in a somewhat clunky fashion.

Things have moved on, and our workFile Vision repository and application takes the next step, staying on the web, but moving out of the browser…

By doing this, we still maintain all of the benefits of distributing an application over the web, however, we also have the added flexibility of running outside of the browser and providing features that can only be made available when running in a “trusted fashion”, such as integrating the web application with thick client applications (take Microsoft Office for example).

In addition, workFile Vision is fully extensible, providing an application framework that allows developers to design new modules and have these plugged seamlessly into the interface. This is to allow the ECM platform to grow with the customer’s needs seamlessly and without developers needing to re-write / re-design modules and applications. Taking this further, all modules can be configured by the user, for example allowing them to design the layout of a repository query form…

Though in the late stages of an Alpha release, workFile Vision 2.0 will deliver everything you would expect from an ECM platform, but much more in terms of the web, extensibility and scalability…Exciting times….I will keep you posted….