ECM : State of the industry

I have just been going through some of my “Friday” reading, and cam across a couple of articles that look at the May AIIM report called, “State of the ECM Industry 2010”. For a look at one of the articles, visit http://www.formtek.com/blog/?p=1331 and read Dick Weisinger quick review of the AIIM report.

Reading this got me thinking a lot more about the actual state of ECM and the businesses that use it, or who should be adopting it.

Drivers behind implementing ECM

There are numerous business drivers for ECM, and I have posted about savings and business drivers on this subject a number of times ( I have a series of posts on True ECM Savings which highlight many business drivers). But what were the “highlights” from the AIIM report.

Apparently the biggest reason to adopt ECM is to optimise business processes, which for me shows the link between ECM and BPM growing stronger and stronger. I no longer see ECM as separate to BPM, rather see the two as a single entity. It is also worth noting that this business driver was with a ratio of 2:1 when compared to compliance…Which is interesting and I believe shows the state of the economy and its impact on business thinking and drivers for investment…

Compliance came in as the second biggest driver for adopting ECM, and this is no surprise. Litigation, regulatory demands, financial reporting, audits and of course fines for non compliance means businesses have to take control of their content in a big way, and the only real way of doing this is by implementing good ECM and BPM practices.

Backing up these two main reasons as that 60% of new adopters cite “Content Chaos” as a business driver in its own right. This is in some ways a pre-cursor to driving for greater efficiency, but does highlight just how problematic varying forms of content can cause businesses, especially as they grow.

Finally, 37% cite “Green IT” benefits of ECM as a driver. Again this shows the state of the economy, not just thinking of the green benefits in the longer term, but how being “Green” can actual make real monetary savings  across any organisation.

The Sharepoint factor

Microsoft Sharepoint gets its own special mention in the report, mainly because it is the new mover so too speak in ECM, though I don’t see this as a pure ECM solution, simply because it lacks so many ECM type functions (again this is something I have posted about in the past). However, it does show that sharepoint has reduced the barriers to entry for ECM, especially at a departmental level.  Apparently 32% of companies have implemented Sharepoint in some way. It is also worth noting that only 11% of these though use SharePoint exclusively as their only ECM solution, probably due to its short comings as an outright ECM platform…

Cloud computing and SaaS

Now this is an interesting area, especially when looking at the barriers to entry into ECM and why SharePoint has seen such massive growth at the departmental level. SaaS really does provide even fewer restrictions to organisations wishing to use ECM, especially at the departmental level. So does this mean with the trend of SaaS that we will see SharePoint face stiff competition at the departmental level of implementations? My own feeling is yes….I have myself already been in talks with organisations that are looking for quick SaaS based solutions, rather than opting for SharePoint.

Apparently in the next 18months the number of companies using SaaS for Document Management (DM) will double to 12% and the number for Records management triple to 6%. I am a little more sceptical about this, especially with Compliance being the second biggest driver for adopting ECM. There are many issues with SaaS, cloud computing and compliance which means for many uses of ECM, SaaS and the cloud will face many issues….However, if you think of smaller uses of ECM at the departmental level, then yes, I can see a massive growth in the use of SaaS for some ECM type functions. Please note some…..

SaaS provides a real quick implementation of areas of ECM. However, it is very restrictive, not just by potential compliancy issues, but also by application integration barriers. If the biggest driver for ECM is business optimisation of processes, then these businesses will be looking to integrate their ECM platform with many other applications within the organisation. Doing this means, you won’t be looking to Cloud Computing or SaaS, as this really does and will limit the potential of integration and therefore the effects ECM can have on business process optimisation. So when looking at the figures in the AIIM report, you have to think of ECM in different ways, with various requirements and regulations….

Open Source

This is a surprise to me, with more organisations looking to open source based solutions. I myself am not a lover of open source solutions, I have always argued that they can never deliver the real security needed for business, and I stand by this. In addition, their cost savings are never as great as initially thought, and there are real concerns with ongoing maintenance and product road-maps. Apparently though, this won’t stop 9% of organisations using open source solution by 2012 (I am not so sure…)

The enterprise 2.0 and Social Media

Though organisations want to be seen as “Enterprise 2.0” many don’t know what this is, or have any idea how to actually go about implementing this lovely idea. Social Media technologies, well their use, is on the up within businesses, you need only look at how many organisations use Twitter and blogging tools on the web. These are key tools to an organisations marketing, communications and PR, however for many they are overlooked as actual organisational content, which is wrong.

29% of respondents view enterprise 2.0 as signification to their organisations business goals, looking at such things as knowledge sharing, collaboration and coordination, making ECM a core technology for them. However, there is a down side to Social media, and that is the negative impact it can have on productivity, with, unfortunately, many employees wasting more time on these websites than actually doing work. The only surprise knowing this, is that only 45% of companies bar access to Facebook, Twitter, You Tube and instant messaging.

A good illustration of the lack of understanding that social media interaction is still organisational content, is that 80% of companies that use Twitter and blogs, do not archive the data, nor have real access to the data from their internal systems (ECM or others).

Last word….

All in all the industry looks set to grow, and there are many newish areas in which ECM can bring new benefits to organisations, social media being the most obvious. However, there are areas where things don’t add up in the AIIM report, the growth of SaaS compared to the quest for full compliance being just one area. In addition, I haven’t read anything about mobile based ECM and the need to access repositories and content across multiple devices, something I feel will become increasingly important in the next 24 months, perhaps more so than SaaS…

No matter what, the next 24 months will be interesting within the ECM market, with lots of new drivers and solutions bubbling to the surface I am sure…

ECM barely being used?

Today’s post has been inspired by some web reading I have been doing earlier this morning. Open Text, (one of the largest ECM players) have made claims that ECM is still only in 20% of the available market, and that tools just aren’t being used by corporations. From my experience, this is probably true, and of the 20% that do have ECM, I would say 10% of these, just don’t use their ECM capabilities correctly if at all.

Is this a worrying fact for ECM?

Well probably not. ECM in some form or name (Document Management, Image Management, EDMS etc) has been around now since the 80’s, however, it is still an immature sector, and still seen as a very niche market place. Often customers of mine state ECM hampered by far too much jargon and far too much emphasis on out of the box connectors across the complete enterprise. This “enterprise” view, can often scare organisations away from looking at the benefits ECM can bring to each department for example.

Back to basics

Organisations and more importantly their users, sometimes don’t get how to use ECM correctly, and it’s not surprising. Many of the users, and business decision makers, will think nothing is wrong with saving files down onto their local machine (or network share), and organising these folders in a logical way. In addition, users when on their home PC won’t really be using anything that resembles ECM, though there are plenty of reasons why perhaps they should…

This is where education of what ECM can provide is really needed. The whole profile of ECM within the corporate world, and small business world, needs to be raised. Because of this lack of understanding or profile if you like, it is often hard to communicate to businesses just what a positive effect ECM would have for the overall running of their organisation. Of course over time, more and more businesses will grasp the many benefits and real world savings ECM brings to businesses, but hopefully this will not be so far down the line that the market place has only a few massive players left.

The large players provide really powerful solutions, and very powerful demonstrations, backed up with great consultancy work and number crunching to illustrate ROI and future efficiency gains and savings. However, sometimes you can take a good thing too far. Often the case for ECM should be a basic one, leveraged at a departmental level, as opposed to the complete Enterprise. By addressing an organisations needs, department by department, you vendors ensure real business requirements are met immediately and that they are not lost in the mass of other requirements when looking at an enterprise as a whole. It’s also worth  noting that organisations purchasing ECM will not be so worried about implementation complications etc. when only looking at an individual department, things are simpler.

Going forward

As vendors, let’s keep things simple, let’s try to raise the profile of ECM and its benefits, let’s try to remove some of the jargon behind ECM and let’s sometimes step back from the concept of a solution for the whole Enterprise. If we do these things, business decision makers will find it easier to grasp the many benefits of ECM and when this happens, ECM will be being used in many many more organisations. Correctly I hope…

Document and file retrieval metadata

Far too much focus is made today on providing complex retrieval fields within ECM solutions, and far too much is made of them from customers. For sure, inherited values and properties can be of great use, but when you start to look at your actual requirements, far too often retrieval fields are simply made too complex.

Points to remember

When designing your retrieval fields, metadata or indexes (whatever you wish to call them), keep in mind just what a user will want / need to do to actually locate this file / document. Here is a quick list to help you:

1. How much information will the user have on a file?
2. How much time do you want to allow them to enter search information
3. How can your metadata fields actually assist in this
4. What sort of results will be brought back and how clear will these be to the user (clear as in how can they quickly see the file they want)

Many systems recently spend a lot of time on very accurately identifying files, however, by doing this they also make it very complex at the data capture stage (scanning and indexing) and also require the user to spend longer setting up their search.

Keep it simple

When designing / identifying metadata fields for files, always try to make and keep things as simple as possible.

First things first, identify the types of files you are storing. This doesn’t mean pdf, word, tiff etc. rather it relates to their type within your business. So some examples may include personnel files, expense claim forms, insurance claim form, phone bill, customer details etc. (dependent on your business).

Once you have made this identification, we get onto the point of retention. How long will a particular file type stay “live”, then move to an “archive” then be completely deleted. When doing this you may find that you logically have some separation of files appearing. NB only create a new classification of file type if it is needed. Don’t do it as some logical separation, rather classifications should only be created to separate either groups of metadata or address such issues as migration and retention periods.

The tricky part is to now identify the metadata fields associated with your types of files. I would always suggest you try to keep these as simple as possible and try not to use more than 7 fields to identify a file. This is where often designers get carried away using inherited fields from different objects within the repository. This is all well and good and can really help in displaying search results back to users (or a heirachyy of files back to a user). However what I try to do is the following:

1. Imagine you don’t know if there are other files out there in the system (nothing to inherit from)
2. Identify at least one key field (policy number, customer  number, telephone number etc)
3. Provide a list of options to the type of file it is (Date of birth certificate, driving license, claim form, phone contract, interview, recorded conversation etc)
4. Only provide other fields that help logically identify this file from other files of the same type, or they help identify, for example, a customer entity within your business
5. Provide as many “drop down list” options as possible. This ensures data is accurate and not reliant on spelling or interpretation
6. Identify any metadata that may be “shared” with other file types. For example a Policy Number may be found on multiple types of files within multiple classifications of files. In addition Policy Number is unique within the business so therefore it can be used to tie together a number of files to a particular policy holder.

If you stick to these 5 principles you will find that 9 times out of 10 you will not have any call for using complex inheritance or complex storage concepts. You more than likely have also identified your classifications in full. Please note that your file types along with classification will also 9 times out of 10 provide you with enough criteria to accurately assign security information to these files.

Once you have identified how information is to be retrieved, think about what information could be automatically captured at the data capture side of things. This sometimes illustrates fields that could be used to help identify files at retrieval; it also sometimes identifies fields that really aren’t appropriate.

Showing results

Often your retrieval system will display results of searches in a format which isn’t always that great to you or your business needs. This is why there are so many “professional services” offered to customers of such systems. As a customer, linking objects together, even showing them in a “tree view” type fashion can help the end user. However, this isn’t a call for inherited properties, rather a call to logically display business related information.

Also remember different types of searches can require different ways of displaying search results. This is sometimes overlooked by designers and system providers to the detriment of the user experience.

Finally, always think past the retrieval process. Once a user has found the file they want they will need to interact with it in some way, this could be to simply view its content or to pass on to another user etc.

Conclusion

I am a firm believer in keeping things as simple as possible and often adopt that IT term the “80 – 20” rule. Far too often IT tries to deliver too much, and in doing so it over complicates areas of the system or worryingly the business. When this happens more often than not a project can be seen as a failure, when really, by delivering less the customer gets more.

When putting together metadata for the retrieval of files remember to try and keep things as simple as possible. Identify key fields and not get carried away in capturing too much retrieval data. Also, always keep your end user in mind, so that’s the end user at the scanning and index stage and end users searching for files. Sticking to these simple rules will ensure you deliver a file retrieval system that works efficiently, quickly and well for your end users and your business…

True ECM Savings…#5

This is my penultimate entry in this series of posts, and in this post I will be looking at Content Security in terms of not only access, but what happens to content in the case of flooding or fire (something that is often overlooked).

Flooding and Fires…

Not the nicest of titles, but it’s something every organisation must think about, “What happens to our content if the whole building goes up in smoke, or we are flooded out?” This is a question that is more often overlooked than you may think. I have visited many “large” organisations that really haven’t taken such disasters into consideration.

When storing files (especially paper) you will be amazed how often fires do crop up, simply do a search online and you will find examples of fires destroying organisations documents and content, including governmental records. A great example of disasters destroying content can be found looking at the after effects of hurricane Katrina. Warehouses full of content and documents relating to criminal prosecutions were lost, leading to hundreds of criminals being released, simply because the content couldn’t be retrieved electronically.

Now think of the actual cost to your organisation if you lost all of that content and documents. Not only may you be looking at issues regarding compliance, but no doubt massive costs will be incurred not to mention potential loss of business.

If all your content and documents are stored electronically, within a good ECM platform, these issues just aren’t there. Sure in some cases you may still want to store the physical paper, but this can be done off site at dedicated centres (outsourced). You still have access to all that content, even if the physical paper is destroyed. You can also distribute your backups of content easily; having backups at multiple sites ensuring that content is never lost.

And to think, I haven’t even mentioned theft of content…

Our content is secure without ECM?

Well no it isn’t. Paper is the most insecure form of storing content, think, if I can get physical access to the location of a file, anyone can read it, or worse, photo copy it and re-distribute as they choose. It really isn’t hard to open a file cabinet, pull out a file and start reading. Content security is more than just ensuring the office is locked at night, or having locks on the HR file cabinet.

It’s imperative that content is secured, in many cases for compliance, but in general, you cannot have employees looking at information they should have no access to. Think of the issues that may arise, loss of business to competitors, stolen ideas, staff suffering identity theft, I could go on.

Though many of us have watched Hollywood films with computer hackers gaining access to lots of sensitive information, and many of us have read about online hackers gaining access to our personal details, the reality is that electronic content is far more secure than paper.

With a good ECM solution, your content is secured in a number of ways, allowing you to grant different levels of access to content, dependent on individuals or their roles. You can also track just what files have been looked at and or any interactions with that content a particular user may have. No one can tamper with or replace a file, leaving you with false documentation. Such ECM solutions ensure content security is therefore controlled by the organisation itself, and not left open to any form of abuse.

Conclusion

ECM speeds up the way in which people work, by providing them with access to content when they need it. This remains true in the cases of natural disasters and theft. ECM also protects your organisations content and access to that content, reducing many content related risks.

This post can be seen as a post about highlighting the “potential” savings of ECM in times of crises. With this in mind, think of ECM as content insurance. You hope that none of these situations arise for your organisation, but if they do, you will be safe guarded and rewarded for your prudence, and save a lot of time and money.

Deploying ECM across the enterprise

I was keeping the world up to date with my day on Twitter, when I read quite an interesting article based around an organisation looking to invest in ECM and deploy it in one lump across the whole of their enterprise.  The article was looking at the “main” players in ECM, so Oracle, IBM and EMC Documentum. It really highlighted the problems these companies had in pitching to the client, demonstrating their product and trying to show how this would work across the complete enterprise…

Having worked with all of these companies in some form in the past, I remembered just how great their platforms are, but also how heavily entrenched they are with marketing and hype. Putting together a demonstration was never a 5 minute job!

Deploying across the enterprise

This is a lovely idea, but in practice unbelievably hard to achieve (don’t listen to sales banter). I really don’t see how it can work well.

Let’s look at some of the basic challenges of deploying a single ECM solution across the complete enterprise in one go:

1. Scale – If your enterprise sprawls across the UK, or even Europe and the world, think of the challenges you have of implementing the system, allowing access and dealing with distributed performance…None of these are show stoppers, but remember a lot of logistical work will be needed
2. Training – Ok, how you going to train hundreds maybe thousands of users for a go live date?
3. Individual requirements – different parts of your business will have different requirements / needs from their ECM platform and BPM
4. Administration – again logistical challenges
5. Support – you need to have in place vast support services

None of these points will stop a project; however each one requires a lot of thought, a lot of processes to be put into place and more importantly a lot of people with drive to ensure everything runs smoothly. However, point 3 (individual requirements) is potentially a show stopper, and it is this point that the large players in the ECM platform try to address with management, configuration, integration and mapping tools. These all demonstration well (when the sales agent gets it right) but actually require a lot of “professional services” to get them working to meet your actual requirement.

Delivering for everyone

It looks great in a demonstration, the presenter simply clicks on a wizard, answers some questions, fills in some datafields and hey presto, your system is integrated and reading in data from a third party. Wow. Likewise, the presenter clicks on a nice processing map, drags some icons onto the screen, joins them up, again ads some datafields and hey presto, you have a workflow….

Now this does look great in a demonstration. And in simple cases, this will work for you. However across the complete enterprise? Will it be flexible enough to meet everyone’s requirements? Are the simplish points of integration shown so well in a demonstration going to work like that for your organisation…….I am guessing a strong NO here is the answer.

These sorts of tools are great for demonstrations, even great for very simple integrations and maps, however the price you pay for such tools far outweighs their actual benefit to your organisation, unless of course you leverage some “professional services” to ensure the system meets your businesses complexities.

The investment…

So to achieve a massive roll out of ECM across your enterprise, you are looking at a massive investment in both time and financially, and then no doubt you will need to address individual units of requirements….All of these factors make it harder for your solution to succeed and deliver that promised ROI.

How would I go about things?

Well first things first. My ECM platform / purchase would not include fancy integrator and mapping modules. For me these add vast costs to the initial purchase and licensing, on top of which you have to purchase additional “professional services”. In my experience, it would work out cheaper to just pay for “professional services” to develop the integration (for example) with your other systems from scratch. Essentially this is more often than not what happens under the term “Professional configuration services” or something similar. However, you have also made a purchase for that integration module license in any case….

This is one of the reasons why I stopped working with the big ECM players, and decided to invest time and resources into our own ECM platform workFile (www.workFileECM.com) We have not wasted time nor money investing in complex integration tools and modules, that look great in demonstrations, but fail to deliver real business benefit. Rather, we develop the integration you require specifically for an organisation, meeting 100% their requirements, based around our open XML Web Services API (something you should insist your ECM provider provides). The same applies to our business process maps, workFile utilises the development platform of Visual Studio to design process maps. Why? Well a developer has so much freedom here, so much so, they can code complex business rules, algorithms, calculations, integrate with numerous other systems and make the workflow work seamlessly for the end user, basically ensure the process map is the power behind the actual solution.

Secondly, look to deploy your ECM unit by unit, or department by department. Each department will have a different requirement, and each department requirements need to be looked at seriously. An enterprise wide ECM solution will only work if each department takes it on board and uses it correctly. This is only going to happen if the system meets their requirements and is championed by the staff.

By implementing ECM unit by unit, you ensure that requirements are not lost, you ease the load of training, administration and support, while easing new processes onto the organisation a step at a time.

 

Conclusion…

By working in a unit by unit basis you not only identify all the requirements needed across the enterprise, but you also ease your implementation headache and keep costs down (often removing the need for fancy enterprise integration modules). For sure, your ECM provider will try to make you go for a “big bang” implementation across the complete enterprise, and no doubt, show you some wonderful tools that make it all seem so easy. But there is nothing stopping you striking that enterprise wide deal, then addressing the implementation on a unit by unit basis, re-negotiating cost if needs be as you go.

Remember fancy demonstration tools may look great and promise the earth, but almost always won’t meet 100% of your requirements. So you need to know what you are looking at for “professional services”, again on a unit by unit basis…

Successful document capture…

Well this is something close to my heart. My first ever project after leaving university was to help write a document capture application that was built on-top of the FileNET Panagon Capture platform. Ahh happy days…Though I did seem to earn the name “scan man” from then on, which wasn’t so great, as every document capture project our company then had, I had to be involved with….

Ok so how do you implement a successful document scanning / capture solution. Well it’s very simple, follow these 5 guidelines and you are well on the way.

1. Throughput is everything. Make sure people can load the scanner and let it do its thing. You don’t want to be stopping to separate documents or batches. Make sure your software can do this and purchase a scanner with a big document holder.
2. Ensure you maximise the quality of the images you are capturing. If this could be a problem, then make sure you get in place good quality control and re-scan technology
3. Identify as much information as possible up-front with your software. The more a user has to do, the slower and more expensive the process will become
4. Ensure your data captured or assigned to a document is accurate. Remember your retrieval of these images depends on the accuracy of your data capture
5.  Your document capture is pointless, unless you release the images into your storage repository with all the correct information. Again make sure this is done seamlessly and accurately. The longer the files are in your capture process, the longer it will take for them to turn up in a customer file for example…

 

So where to start?

Well this is with your document capture software, and there are lots of solutions out there. Firstly, when choosing your capture software, have those 5 guidelines in your mind. You want to automate as much as possible (unless we are talking only the odd scanned document through the day). In addition, you don’t just want to watch a sales pitch on the actual scanning process, or the physical scanner being used. You want, and need, to see the process all the way through, and with a variety of documents.

It’s best if you can use forms wherever possible, but you will always have un-structured documents coming to you, such as letters. Now you MUST see a demonstration of how these are dealt with, then ask yourself;

“is that efficient?”

“how could that be speeded up?”

“am I happy with the way data is entered / captured?”

“now let’s find the document in the retrieval system”

I don’t want to start recommending software, as depending on your storage repository etc you may find you have a limited selection. What I will say, is that for our workFile ECM repository we use software that I have been familiar with and more than happy with for sometime, Kofax. I have worked on numerous projects with Kofax Accent Capture and with Nuerascript recognition modules (which are now part of Kofax). Kofax provides you with all the technology and features you could want to streamline any capture environment. And, more importantly, they allow you to write your own release processes into the repositories of your choice.

What about architecture

Scanning can be quite intensive for your PC. A while back, all of your “steps” if you like were carried out on a single machine, so you scanned, had the batches and documents recognised, processed, enhanced then sent on for an agent to index. However, this isn’t great, ideally you want to split out this intense processing work and let your scan station simply scan images.

Server based solutions are best, freeing up staff to scan and pull documents as and when they are ready. Your images should always be ready quicker than your staff can quality assess them or carry out indexing tasks. Oh, don’t be fooled by “thin” document capture, something has to drive the scanner and therefore it’s not “thin client”…

What about staff?

This can be a boring task, so rotate your staff to different jobs, every couple of hours. They may still get bored, but if you don’t do this, they will be making lots of errors and getting really bored. Trust me, just spend a couple of hours doing one task such as scanning and your brain can go numb…

You will also need a “champion” of the capture process. Someone who can keep people motivated and ensure they maximise the potential of the system. All too often the system capacity is not met as staff becoming lazy or complacent. This negates your investment and diminishes your return on your investment, so a champion is very important.

It’s also worth noting that from time to time, you will need someone with more experience of the scanning process, again that champion, simply because you will get issues with stuck paper, batches not getting recognised, image quality problems etc. At this point, you need someone with a little more knowledge of how things work.

 

Finally

Remember no matter how good your capture process is, your retrieval system is only as good as the quality of the images and the data associated to those images. Also, please don’t invest heavily in a great capture system then scrimp on your retrieval system. If you do this, you will find no benefit of the capture process and document imaging at all. Your first port of call is still ensuring you purchase the right retrieval / document management system. Then address the capture side of things.