Big data has been with us for ages having existed and getting churned out in both structured and unstructured formats, only that it existed offline or limited distribution digital. With Moore’s laws that says processing capabilities for computers doubles every twenty four months still holding coupled with an equivalent drop in the cost of storage and access to better connectivity the uptake of the cloud has seem tremendous growth of big data. Essentially unshackling the traditional models from constraints which included curation, analysis, search, visualization, storage, sharing, capture among others.
Historical data has been ingested offering unique insights into the past and a possible peek through extrapolation and modeling into the future, but going forward a new itch needs to be scratched. The dynamics of life driven by consumer demand have birthed new services that require not just big data but fast data. The enterprise in an increasingly competitive environment needs real-time and on demand business intelligence and analytics to guide decision making, service personalization, service provisioning and resource allocation.
The need to be both nimble and agile requires that we look beyond the 4V’s on which big data is hinged; volume – the quantity of data generated or ingested, velocity – the rate at which the data is created, veracity – quality of the data and variety – different flavors and formats. Processing and decisioning is now the competitive focus with data warehousing and smarter ingest management in supporting roles.
Where does fast data apply, you may ask? Think of the backend that would power a pan-african mobile ad serving platform with millions of impressions, multiples devices and varied ad formats served in real-time across different geographies. Imagine the innards of a country wide mass transit system serving millions without missing a heartbeat from payments bring made, cards being issued and topped up and scheduling of the transports. Closer home, visualize a humming traffic control ecosystem that uses data from mounted cameras and a range sensors on the road and on vehicles. Fast moving consumer goods fulfilment systems that are mapped action at the till and market activation initiatives to allow for scaling up or down of production and logistical adaptations to achieve maximum return on capital. Lots of data collected, but only of value if processed and output blazing fast.
The real-time intelligent enterprise is here.