Category: Uncategorized

Business Process Optimization using Big Data Solutions

As part of my previous post, I had explained the benefits that Utilities & Telcos can experience if they leverage the power of Big Data towards making their Business Processes & the participating systems more efficient. As part of this blog, I will be concentrating on the first of the 2 variables (i.e.) the Business Processes and how Big Data can help make them efficient.

To justify my stance, I will be concentrating on the following 4 components of a GIS driven Business Process:

  1. Business Process Flow: To identify phases of the Business Process that are potential bottlenecks.
  2. Data Quality: To identify common data errors which might be resulting in lengthy digitization cycles.
  3. Back-end Tasks: To identify any back-end processes which might be delaying the Business Process cycles & lastly
  4. System Integrations: Time consuming integrations, Web-services (for example) , which invariably elongates the time taken for Business Process to be completed.

4 components

With that in mind, I would like to propose that we investigate the title of this blog by using a use case, an imaginary one. Here is how the story goes:

Client A (an Electric Utility) has a few GIS driven Business Processes and they have invested in the Big Data based monitoring tool.

Now, before going into the “how” of it, I would like the reader to know that the Monitoring tools that I have mentioned in this blog are capable of tracking everything that is happening within the GIS system. So they are capable of tracking things like:

  1. How much time a particular Business Process has spent on a particular phase (Ex: how long has a given New Connection business process been in the In Design phase?)?
  2. What are the errors that are being reported frequently and for which objects and in which Business Process?
  3. How much time does the execution of a given batch process/web services takes and how many times it has failed? and so on.

Lastly, just a FYI, the dashboards that I have used in the following sections were created using Diagnostics running on top of GE Smallworld. Also, I have tried to keep the explanations as minimal as possible.  This is because I have this slogan which I go by:

“If you have to explain a dashboard, it is not much of a dashboard”

So the assumption here is that these dashboards are self-explanatory and the reader, regardless of his system know-how, should be able to make sense out of them without assistance.

Enough chit-chat, lets look at how Client A is able to make their Business Process efficient using these monitoring tools.

Step 1: Knowing where to look

The first step towards making their Business Processes more efficient is to know the following 2 things:

  1. Which are the most used Business Processes?
  2. Which of the most used Business Processes have long completion cycles?

To do this Client A opens up Diagnostics and fires up a queries to create dashboards to identify the most active and the most time-consuming Business process. They then use these dashboards to identify the Business Process that they want to Focus on. In this case it is the New Connection Process (refer slides below).

This slideshow requires JavaScript.

Insights 1

 

Insights gained from above two dashboards helps Client A to get an understanding on the Business Process that they need to focus on. Since the analysis phase consists of cumulative insights, I am going to keep a track of the insights obtained in a notepad as seen in the adjoining image.

 

 

Step 2: Where are the bottlenecks in the New Connection Business Process?

Once Client A understands where to look, the next step is for them to understand which of these. Here is how it looks:

stages

insights 2

 

The above dashboard provides details on the average time spent on each phase of the New Connection Business Process. From the above dashboard Client A can conclude that the bottlenecks are possibly present in either the “In Design” (or) “Post Pending” phase of the business Process. Lets update this insight to our notepad and move on.

 

 

Step 3: Identify Bottleneck Causes

Having identified that the In Design & the Post Pending phases of the Business process are where most of the time is being spent, narrows down the scope of potential bottleneck causing candidates. We can sum them up as follows:

Potential Candidates

insights 3

In summary, there are 3 prime candidates to investigate for Client A to understand what might possibly be contributing to the performance bottleneck in their New Connection Business Process. Lets make a note of this in our notepad and move on to investigate these candidates one by one.

 

Candidate 1: Data Validations Checks

Data errors are one of the leading contributors to lengthy digitization cycles. Ensuring that the users make minimal data errors during digitization not only helps shorten the digitization periods but also contributes directly to the confidence that a Utility might have one their GIS data.

So, to identify get insights on the QAQC errors related to the Business Processes, Client A fires up the Data Errors & Validation dashboards & focuses on the ones with a significant contribution to the performance pie. This is how it would look like:

This slideshow requires JavaScript.

insights 4Dashboards such as the one provided above, helps the client to identify what are the most common type of errors that are being reported on the digitized data. They also help provide the first actionable insight for the clients. In this case the action item can be for Client A to provide training to their network engineers focused specifically around reducing the Phase & Voltage Check related errors.

We add this to our notepad and move on to the next candidate.

Candidate 2: Back-end Processes (or) Jobs

Next up Client A turns their attention towards the scheduled jobs (or) batch processes which gets executed as part of the transition of the design. To get insights into the possible bottlenecks they create the following 3 dashboards using the monitored data from Diagnostics: 

This slideshow requires JavaScript.

insights 5Again focusing on the ones that occupy the top slots in the dashboard they understand that the New Connection process has a lot of errors being reported while triggering the Generation of PDF & Submission of BOQ (Bill of Quantity). They also understand that most of these errors are being reported while Posting the data and while requesting the Approval of the designed network.

We make a note of these insights and move on to the last candidate.

 

Candidate 3: Integration related Web-Services

Lastly, Client A looks at the following integration related dashboards available in Diagnostics. Narrowing their focus on the top contributors they arrive at the following insights:

  1. Link assets, one of the most frequently used web-services, takes a lot of time to execute.
  2. The load balancing among web-servers is not optimal.

Webservices 2

They make a note and wrap up their analysis mode (final notepad provided in the next section).

Analysis Outcome

Looking at their “Analytical Notepad” Client A now knows exactly what they need to do in order to make their New Connection Business Process efficient. Here are the action items that they can focus on:insights 6

  1. Reduce Digitization cycles by educating the users on how to avoid the Phase & Voltage related errors.
  2. Reduce Backend Process timelines by focusing on making the jobs associated with PDF Generation & BOQ Submission efficient.
  3. Diminish Integration Bottlenecks by making the Link Asset web-service efficient and by optimizing the load balancing configuration on their Web-servers.

Summary

Big Data driven monitoring tools can help deliver actionable insights using easy-to-understand, data driven dashboards. Utilities & Telcos can leverage these insights to pin-point possible performance bottlenecks to focus on in an effort to make their Business Processes efficient.

Benefits of looking within with Big Data.

Introduction

Big Data, Artificial Intelligence, Machine Learning, etc. are the buzz words in today’s digital world. There are thousands of articles online advocating ways in which these technologies can help benefit organizations who choose to adapt it. These benefits vary from making grids more efficient, predicting customer churn to improving operational efficiencies.

bees_big_data

 

I am GIS guy who primarily works with Utilities & Telcos. My field of work has naturally made me incline towards the understanding Big Data and the roles it can play in #GIS #Utilities and #Telcos. Despite searching across the web intensely, I found that very few of these articles had answers to “How Big Data can help make GIS solutions efficient?” Upon pondering, I understood that the most of these articles on the web talk about how companies employ Big Data in improving their “external sections” of the Utilities & Telcos, whereas the solutions that I deliver help sections “internal” to these organizations.

External & Internal Sections

Let me try to explain what I mean when I say “external” & “internal” sections of an organization. When I talk about making the external sections more efficient, I am referring to the services that the organizations offer to their customers.

This includes

  • Ensuring low down time whenever there is an Outage,
  • Offering Tailored services to Customers by analyzing their behavior,
  • Any other services that might be offered to make the Customers happy.

So basically “external sections” means the services offered by the organizations with an objective to enhance the customer’s user experience. While “external sections” focus on the deliverable, the “internal sections” are things that assist the Utilities & Telcos in these services.

Internal sections can be

  • Efficient Business Process cycles
  • Identifying and removing the performance bottlenecks of GIS systems or  any other systems that participate in Business Processes.

That covers the “what” of it. Next up “Why”.

Why is making internal sections efficient a Big Data problem?

Any problem can be classified as a Big Data problem if it involves the 3 Vs (i.e.) Velocity, Variety & Volume. As I mentioned in the above section, the two primary contenders for making internal sections efficient are improving system performance & building efficient Business Process.

Improving system performance is traditionally done either by using profiling (source code analysis) or using system logs. While profiling adds substantial overhead to the already bad performance, use of system logs is a classical Big Data problem. Here’s why:

a. Average size of the log file generated from a regular  user session is about 5-10 MB (Volume)

b. Almost no two user log are similar (Variety) and

c. Logs are generated every day containing different/new information on what the administrator is looking into (Velocity).

Similar framework can also be applied for cases where organizations, either proactively (or) re-actively, decide to improve their business processes. They have to involve numerous stakeholders to understand where the bottlenecks are (volume & variety) and they have to analyse user system logs to validate their claims (velocity).

That leaves us only with the “how?” and therein, as the bard would tell us, lies the rub (reference for the naive). Read on.

Big Data Tools for addressing internal section problems

There are tools available in the market which are designed specifically to address the internal section problems of an organization. For example, there is a Diagnostics for organizations which use GE Smallworld as their GIS platform, ArcGIS Monitor for organizations which use ESRI.

Some of the salient features that these tools share include:

Intrusive Monitoring

These tools bind closely with the GIS solution and are hence able to track almost everything that happens inside the GIS application in real-time. This means they allow real-time tracking every single user click, every database transaction, frequently/rarely used tools, time taken to perform a particular task or to execute a particular query, etc.

Negligible Impact on the User Experience

Intrusive monitoring does raise concerns of these tools adding additional performance overhead to the GIS systems that they are monitoring. Most of these performance motoring tools, at-least the ones mentioned above, are designed to ensure that they have little or no impact on the user experience.

Here is a two minute video of me explaining how Diagnostics makes it possible.

Easy to understand results

These tools leverage the power of Big Data to crunch the rapidly flowing stream of humongous real-time data and make the results available to the administrators in the form of easy to dashboards. These dashboards allow administrators to identify bottlenecks in their GIS systems (or) business processes and take necessary steps to resolve them.

Pro-Active Maintenance

Ability of these tools to monitor almost everything that is happening inside the GIS solution in real-time allows administrators to be pro-active in their maintenance effort. For example, administrators can configure these tools to send them email whenever the size of the database reaches a particular value so that they can extend its storage capacity.

Integrated Monitoring

Since all of the processed monitoring data are made available in a single platform, it allows administrators to not only identify bottlenecks but also to understand tools/processes which are getting affected by them. So for example, if the administrator is able to identify queries which take a lot of time to execute, he can then go ahead and also find out tools which use these queries or they can go a step further and identify Business Processes which are getting directly affected by these queries. These information can then help justify any performance enhancing investments made by the organization.

Summary

Utilities and Telcos can gain massively by actively investing in solutions that help turn the beam of analytical prowess, offered by Big Data, towards their internal sections. Some of these organizations, having identified this opportunity, have already began making such investments. This is a brilliant step forward by these giants as this promises to illuminate many-a-previously-eclipsed performance snags, thereby resulting in happier users, smoother and more efficient business processes.