Edge to the Cloud – A data odyssey!

Let us imagine the year 2021. We see not just flying planes but flying people and flying drone traffic. And yes they fly on their own very smartly without colliding against each other and dropping off the sky. Least to say the roads are packed with smart cars of all sizes. Interesting isn’t it? But this is certainly not science fiction, it is going to be a daily fact of life.

Flying_City

How will this happen? This will be a completely AI driven planet. Machines that will not require human intervention to perform their mundane operations. They will communicate with each other and will make smart decisions on their own.These smart machines will have inbuilt collision avoidance and image recognition systems. These systems will have running convolutional neural networks and long short term memory network models running in their embedded systems. There will be real time capturing, process and actionable output driven activities generated. All in one smart decision support system.

However these cognitive flying objects will require huge amounts of real time data processing. To process this huge amount of  big data objects the systems will have embedded GPU systems in them. The challenge will be that this kind of real time processing of AI algorithms will not be possible in the cloud due to wireless latency to transfer data even with 5G systems deployed everywhere. So most of the realtime AI operations will have to take place in the embedded systems of the flying machines. Once the machines are backed to their docking stations that is when all the flying data will be archived to the cloud. The cloud will also be instrumental in controlling fleets of these smart vehicles and storing data for historical analysis. It will also be the central brain for large decision support systems.

This real time data processing at the source edge and storage in the cloud is going to be called edge to cloud networks(E2C). Architecting and developing systems that will support optimized E2C operations will be key and frameworks have to be defined. But indeed these are going to be interesting times for distributed system architects.

E2C_Architecture

Figure 1: This figure explains the high level E2C architecture.