Tag Archives: Wiki

Fear of Open Source

Big companies tend to go for big companies such as Microsoft or Oracle when it comes to supporting their IT needs. The main reason is exactly that : support. Your contract will always ensure that whenever something goes badly wrong with your systems and your internal support can’t handle it, then there’s always the option of big brother giving you a helping hand.

So far, so good.

Most of the time it doesn’t really matter too much which system you’re going for, provided you have a sensible balance between fitness for purpose and a small enough number of systems that can be supported. But what I’ve seen happen over time is that anything that has a whiff of being “non-standard” gets the evil stare.

Take for instance the Hot Mill Systems guys at Port Talbot’s hot strip Mill, who for reasons that were hidden in the mists of time (but probably had something to do with the fact that Martin Doyle was more familiar with Linux boxes and the languages such as Perl and php that go with it) did not fit in with the Process Control set-up of the time which consisted of Windows servers, Visual Basic and ASP.

At one point someone from the hot mills management team had been approached by the Process Control manager to see whether a switch from their open source set-up to the standard Process Control could be considered. That question was passed on to me, to which my reply was : “Do you realise how many thousands of web pages alone would have to be rewritten in order to make the conversion?” Not that it would be impossible to convert existing php pages to ASP, but it would be a mammoth task that would take a whole team several months just for the sake of tidiness, with no new functionality to show for it.

Mind you, at a later stage when I was asked to rewrite the morning meeting pages I did so in ASP.NET, not only because I had got out of the habit of writing web pages in php, but also because the added functionality I could achieve in .NET. However, the decision was made on practical rather than ideological grounds.

On the other hand, if you spoke to Process Control, and even more so to the business analysts in a GIS, the whole concept of open source was complete anathema, and to be avoided at all costs. Maybe this was because they were system administrators but not coders, and maybe felt that they might be out of their depth if there was not the security of a well-supported back-up solution.

This knee-jerk aversion to open source became clear to me when I was considering whether it would be possible to change from our existing Sharepoint wiki to MediaWiki which, since it is the software supporting Wikipedia, I considered to be the industry standard. At the time I was looking for a wiki solution that might be more user-friendly than the Sharepoint set-up, and thought that the ease of adding images, as well as the availability of discussion pages and automatic indexing at the top of an article made MediaWiki a strong candidate to migrate to.

So I put some feelers out to see whether there was any chance to try MediaWiki out as an alternative to our existing Sharepoint wiki. No sooner had I broached my proposal that the shutters came down – in the analyst’s words: we don’t do open source. No point arguing, it appeared to be a dogma that never mind how useful a piece of software might be, if it was open source, it did not get past the first hurdle. End of.

Never mind that in my opinion MediaWiki IS the standard when it comes to wikis, because it is open source you could not even get to the point of discussing the relative merits and demerits of the old and the proposed solution. Which is a real shame.


Excel Is Not a Database (Part 2)

Below is the content of the wiki article entitled “Excel is not a database” as it was at the time I left Tata Steel. There may be some overlap with the contents of the previous blog, but emphasizes the various bullet points where Excel falls in my opinion short of being a satisfactory solution for business reporting. Some items are repeated under different headings, meaning that their bad influence is multi-pronged.

Excel spreadsheets are easy to use and are an ideal tool for ad hoc reporting and investigation of data. However, the fact that it has so many tools, including macros, at its disposal and can hold large amounts of data has misled some people to think of Excel as a database. This is a fatal error, because it leads people to think that it’s OK to employ Excel (and any other type, for that matter) spreadsheets for business reporting.

Below is a shortlist of various reasons why those using Excel as a database are badly mistaken:

Localised Copy

  • Unlike a web application, Excel is a one-user-at-a-time application – it’s not sharable unless you email it (Modern Databases Are Naturally Multi-User)
  • Large Excel spreadsheets use up RAM in a way that interrogating a database doesn’t (Databases Can Work with Database Files Much Larger Than Available RAM)
  • Information silos caused by localised nature and need for in-depth knowledge of the spreadsheet (super-spreadsheets need super-spreadsheet experts)
  • Encourages silo mentality and data hiding


  • Unlike a web application, Excel is a one-user-at-a-time application – it’s not sharable unless you email it (Modern Databases Are Naturally Multi-User)
  • Large Excel spreadsheets use up RAM in a way that interrogating a database doesn’t (Databases Can Work with Database Files Much Larger Than Available RAM)
  • Encourages silo mentality and data hiding

Data Integrity

  • There’s no centrally controlled master copy with the officially sanctioned data, and anyone can adjust their copy to their liking
  • Data errors are not corrected at source
  • Correction of identical information that exists in multiple places
  • Multiplication of spreadsheets for multiple reporting purposes
  • Manual entry without error checking
  • Data trail from raw data to final report becomes confused
  • Arguments over what constitutes the “real” figures
  • Multiple hits against data sources for individual reports that have large amount of overlap

“Sophisticated” Spreadsheets

  • Fancy Excel spreadsheets require you to write macros in VBA – you may as well spend that time learning how to program properly on a database
  • “Sophisticated” spreadsheets become impenetrable to the uninitiated
  • Information silos caused by localised nature and need for in-depth knowledge of the spreadsheet (super-spreadsheets need super-spreadsheet experts)

Static Copies

  • The data layer and the presentation layer are inextricably linked – if you want an additional column for some calculation, you need to physically create that column
  • Excel doesn’t have inherent back-up/restore capabilities
  • Lack of drilldown facilities, so that in meetings the reported figures can’t be interpreted
  • Summarising daily figures into weekly figures into monthly figures etc. without manual intervention
  • Snapshots miss subsequent alterations to the feeder data

Manual Effort

  • Spreadsheets always need someone to look after them
  • Multiple hits against data sources for individual reports that have large amount of overlap
  • Summarising daily figures into weekly figures into monthly figures etc. without manual intervention
  • Updating of super-spreadsheets can take a long time – reports may take days to be available to decision makers or otherwise massive manual effort needs to be made to speed up the reporting phase
  • Different contributors often need to be chased to update the spreadsheet
  • Data specialists should spend their time analysing and understanding data rather than creating and fine-tuning spreadsheets (= opportunity cost)

And there’s a lot more where this came from under “Other”, but I suppose you get the gist …

Unfinished Business

When I retired at the end of March 2016, there were a few things that I had hoped would be finished but weren’t. I’m glad to say that none of those were under my control. The ones that did I had completed on time, like the conversion of all VB to .NET applications (necessary for the future proofing in Windows 8.1), the hand-over of the Occupational Health and internal audit systems to Tata Consultancy Services (TCS), and the documentation of various systems in the Strip UK wiki.

However, there were a number of projects specific to Strip Products UK that should have been finished by the time I left and weren’t, thereby making it impossible for me to make sure that my applications and systems were futureproofed for them. In increasing order of importance they are :

  • The replacement of the CADS complaints recording system with the new Focus CRM;
  • The replacement of Windows XP machines with Windows 8.1 laptops and desktops; and
  • The completion of the Supply Chain Transformation project.

I had a whole suite of applications extracting complaints information from the mainframe tables CADSDETS, COMPMIS and CREDMIS, which formed the basis of an all-encompassing complaints reporting system. The new Focus CRM had already been introduced for certain aspects of the complaints handling system, but had not yet come to the point of superseding the old mainframe recording systems. If it had, then I would have been able to turn off about 10 applications, since the new systems would no longer require my input.

Tata Steel in South Wales and before it Corus had been using Windows XP for quite some time (although I remember using Windows NT during my first years in Llanwern, and after that Windows 2000 on some machines for a number of years), but the time had come and gone when XP was no longer supported, and Tata Steel had already used up the first year buying support while we were getting ready to convert to Windows 8.1. I had already acquired a laptop to test out various portions of coding and installation of certain types of software in the new operation system, and when Theo joined to take over, that became his de facto workstation.

Still, the introduction of new machines happened quite late and started with the lower end of the requirements scale. Since I was a high-end developer and would be one of the 5% who would receive a desktop rather than a laptop or a thin client, I knew I was going to be at the back of the queue, and that part of the queue hadn’t been reached by the time I left the company.

As for Supply Chain Transformation (SCT), it’s beginnings are so shrouded in the mists of time that I can’t quite remember whether it was launched in 2010 or 2011, but it was a long time ago. SCT was a new way of placing orders and performing order tracking through the manufacturing process, but the problem started with the fact that our mainframe-based Manufacturing Execution System (MES) had to interface with this new system of handling order and customer details.

A further problem with our MES was that its data were set up for running orders on individual production units, but were not really that good at performing through-process tasks. On top of which, the data quality in the MES systems was not always as watertight as it should have been for through-process tracking.

The result ? Every half year or so the implementation was put back another six months, and when I left the company the date was set for June 2016. Whether this really was going to go ahead is another matter, and one I don’t have the answer to. All I know is that Stuart Wilkie said he was not going to give the go ahead if it might bring the company on its knees.

In fact, there’s so much more to say about SCT that I probably will keep it for a later blog. The consequence of these three bits of unfinished business is that I was not in a position to do a clean handover, and although it pains me, I had no option but to leave people to their own devices when it came to these three topics. Although maybe I’m being too harsh on myself, because no one has sent me a cry for help in the intervening months.

I Don’t Do Attachments

“I don’t do attachments”, those were reportedly the words of Ian Hobson when he was Operations manager in Ebbw Vale. Meaning that if you sent him an email, you had to state what you had to say in the main text of the email without reference to any attachments, because he would not even look at them. At the time I thought that was a rather odd thing to do, but over time I’ve come to understand his point of view (at least if not carried into extremes).

The reason why I’ve come to this understanding is Lotus Notes databases. During the British Steel and subsequent Corus days, our email system was IBM’s Lotus Notes, something that was only replaced by Microsoft’s Office365 during the Tata Steel years. The Lotus Notes also came with the option of creating “databases”, sometimes properly created and useful systems such as the Next Steps databases or Process Control’s Post-Product Support database. However, most of the time it was merely a vehicle for clunky text-based documents, and worst of all some databases consisted merely of a forest of attachments without a word of explanation of what was where.

That’s why, when I started the Strip UK Sharepoint wiki, I immediately disabled the facility for adding attachments. I wanted a wiki article to show on the first click the information most relevant to that article, explained in normal English and accessible without having to click on an attachment to access information that should have been in the main body of the article to start with.

Obviously, the time and effort spent in getting your thoughts together and putting them in writing is greater than in merely attaching a document, but in the end the usefulness of an article is enhanced immeasurably for any user who visits the site at a later stage.

I’m fairly sure there’s a general message in here, in that a little bit more effort in advance can make most systems more useful in the end. Or am I over-generalising ?

Strip UK Wiki

It all started when Lianne Deeming wanted a document library for the Process Development Group. It would contain copies of and references to reports and documents of interests to her department. A group of relevant people was convened to try and establish what format this type of library should take.

To my horror the consensus gradually converged on a Lotus Notes Team Room. I say to my horror because anyone in the know should realise that Lotus Notes at this stage (we’re talking 2009-2010) was old technology which did not lend itself very well to integration with modern browsers. Despite my protestations the library took the form of a Team Room and as I suspected it disappeared without a trace without ever coming anywhere near realising the purpose for which it was created.

I still had the idea that a library of some sort might be a useful thing to have, and continued to search for alternative vehicles to try. That’s when I found out that IJmuiden were investigating the use of MediaWiki (the software use in Wikipedia). I managed to have a personal space created for myself, and started to place a couple of articles in it. Finding that this was a fairly simple process, I started to read up on how best to use wikis for consolidating knowledge and fighting what I saw described as “corporate amnesia”. The latter is a reference to the fact that companies often have to re-invent the wheel because knowledge is mostly held in people’s heads, which disappears when they leave the company.

I also looked at how the IJmuiden people were trying to use the wiki and saw that they totally missed the raison d’être of a wiki, which is to act as a collaboration tool (hence the motto “if it’s wrong, change it – if it’s missing, add it”). What they were looking for was something like SharePoint or Lotus Notes databases, which allow you attach existing documents, rather than the wiki which forces you to spell out in words how things fit together.

Not surprisingly, given this misunderstanding, IJmuiden decided to pull the plug on MediaWiki just when I, together with a small number of enthusiasts, were starting to get the hang of how to extend the wiki. Fortunately Andrew Griffiths, then with Process Control and in charge of their servers, knew of a stopgap solution which was the SharePoint wiki. The latter comes free with Microsoft servers, so that’s what we settled for, even though it lacked a number of features that make MediaWiki so user-friendly.

That’s probably one of the reasons why the number of active contributors remained rather small, but for this small group it contained a treasure trove of information from what acronyms meant to explanation of mainframe tables and field names. In the end I also used it to create user guides for systems I had developed, which helped substantially in transferring the data systems I had created for the Occupational Health and Internal Audit teams to the care of Tata Consulting Services (TCS).

In the end, however, I failed in my main objective to see the wiki become a mainstream instrument of information useful to the business. Was it the cultural discordance between how Strip Products UK does things and the approach required for a successful wiki ? Or maybe only a small percentage of people feel comfortable using and contributing to a wiki, and in an organisation of 4000 people that limits the pool to a small number of active contributors.

Whatever the case, the Strip UK wiki (it’s unlikely you’ll be able to access this, unless you manage to overcome the firewall) was useful for myself during the time it was up and running – I can’t say what happened to it since then, and how it will fare in the upheaval of future developments. I can only hope that it will remain useful to some extent to the people who remain in the South Wales part of Tata Steel.