Advanced Analytics on the Edge



Advanced Edge Analytics

with Javascript, MongoDB…

Edge analytics is an approach to data collection and analysis in which an automated analytical computation is performed on data at a sensor, network switch or other device instead of waiting for the data to be sent back to a centralized data store. Edge Analytics is a term normally related to IOT.  The paradigm shift is here –  We need to provide Advance Analytics near the source devices… Right on the Edge. Very soon it will include the more standard type of data sources not normally linked to an IOT Device. Are you ready? Is your infrastructure ready? Can you handle large scale compute on your Edge System?

If you would like more information -We can help you get ready, Please Email Us.

Learning More about MongoDB – Javascript

So, This week we spent some time learning a bit more about MongoDB and Javascript: Part I – Installation.  For me and my customers – We want to do a bit more where the devices are located. Yes, The data is still going to the cloud but what about doing some advanced filtering, storage metrics, simple aggregations (count, avg, min, max ..etc.) and more – Right there near the device in the area that is normally reserved for CACHE systems and/or staging storage.

A Simple Example

Just expanding a bit.  Lets say I have a IOT Device (Temperature: 98.6) – (streaming)[via kafka, mqtt] -> Redis or (memcache). Pause there and think about the great metrics we can gather.  Some of the smarter devices will store min, max and avg .. BUT over what time window?  Wouldn’t you want something like rolling average of the last 5 mins, 1 hour, 4 hours or custom time interval. Spending too much on data transfer – how about filtering garbage data out before the transfer.  Do you have a need to processes something in the sub-second response domain? Doing root cause analytics or predictive failure? How much is down time costing your production?

Analysis – Limitations

Yes, You can perform complex analytics in the edge – including running R and Python Data Science Tools. The data sets on the edge are tiny compared to the big picture that may include 1000’s of devices and they are in a constant state of change.  The code that performs the analysis better run fast and produce results as quickly as the data changes.  Yet, it doesn’t mean you have to use a different tool set completely.

Also, There are limitation to Edge Analytics :

  • Data Sets – The data sets are typically local to the field devices and NOT global.
  • Compute – CPU /  GPU acceleration may not be available on the edge.
  • Memory –  usually finite on the edge network.
  • Network bandwidth – Device Messaging vs. Analytics Interaction.
  • And More …

So… “Tread Lightly”. You would not want to saturate your edge network at the expense of loosing a critical data message from a device.  You need to balance resources carefully.

Questions you want to Ask

In an IOT project, You cannot overlook the power and value of performing analysis of real-time data-sets on the edge. This does require you to think a bit more about the following:

  • What type of analysis you want to perform?
  • How quickly you can respond to a metric high or low water mark?
  • How do you compute ROI and Risk for down-time and failure states?
  • And much more…

Final Thoughts…

Edge Analytics is not everyone.  Allot of IOT projects are just fine sending everything to the Cloud and using traditional analytics. However, If your action window is smaller? Your collocated with your production? You have constraints or issues with resources, data corruption, security and others. You may want to consider moving some of the traditionally Big Data Analytics from the cloud and move it on-premise. To make this determination is an Analytic unto itself.

If you’re not sure – Schedule a call [ Email Us ] with us Today. We can help you evaluate you need for an Edge System.

Thank you, – Gus Segura

Please Contact US or Subscribe to our Blog if you found this interesting or would like more information.

Subscribe : Blueskymetrics Blog

* indicates required,  Managed By Mail-chimp – Please check your Spam Folder and Confirm Subscription.