Big Data and AI Markets Shift Focus

Posted by Jeff Pelliccio on Jun 25, 2018 9:00:00 AM

In ICS insights

There was a flurry of new tech last year that is dizzying to think about. The cloud spread optimistically from company to company, much faster than anyone anticipated. Several new tools came with it. AI is now part of every area of life, while IoT and edge computing are new tech we are still getting used to at the moment. IT workers had to get used to terms of the trade: cloud-native, Kubernetes, cloud databases, serverless, and others. 

Tech is great if you're into it, many business owners, IT purchasers, and developers don't really notice this huge revolution yet. People have been slow to see the business value. In the coming year, many trends will emerge that focus on making this new technology easier to use.

Integrated Platforms and Serverless What?

Serverless computing is a newer term in cloud-computing. The cloud provider manages machine resources dynamically. Pricing is based on actual resources used by an application, instead of pre-purchased unit. It's a kind of utility computing.

Amazon and competitors are racing to gain market share,  so the level of cross-service integration and abstraction keeps rising to facilitate the growth of developer productivity and customer buy-in. For example, Amazon introduced database-as-a-service and fully integrated AI libraries at an AWS Re: Invent event. The big data company also started labeling forms of serverless distribution:

  • AWS Lambda is about serverless functions
  • AWS Aurora and Athena are serverless databases

This definition widens the serverless computing concept to all services that add underlying servers of abstraction. More cloud services qualify as serverless in these wider categories. This trend is likely to continue as the market heats up. Cloud providers will emphasize the integration of unique services by adding abstractions. They'll concentrate on data management, AI services, and serverless computing. This makes the jobs of developers easier because it hides the underlying complexities. Once the technology becomes better understood, though, there is a risk of lock-in and customers dependent on this technology are likely to complain if it prevents competition in the marketplace.

In 2017, cloud providers flocked to Kubernetes to provide the microservices layer, and this prevented customer lock-in due to the common dependency on this tool. In 2018, there's been an increase in the number of commercial and open services that sit on top of Kubernetes. The new offerings provide multi-cloud alternatives to proprietary systems. Iguazio’s Nuclio and Red Hat’s Openshift Paas are such multi-cloud platforms.

Intelligent Edge or the Private Cloud

The cloud increases business agility so that data-driven applications can progress. Startups and large corporations will benefit. Yet, there's still the reality that data has weight, like tractor-trailer trucks cruising the cyber highways. As more data is pushed to the edge in cloud computing, will those highways become jammed? 5G data bandwidth has been augmented, but on the other side of the equation, latency, GDPR legislation, and other requirements mean you have to put formulation nearer the data source.

Today, there’s a public cloud model based on service consumption, and users and developers bypass IT. They use serverless functions and upload video to the cloud.  However, the services that must be constructed in-house are dependent on technology that's still evolving quickly. This makes it almost impossible for IT teams to build services and forces their organizations out to the cloud.

IT vendors offer solutions packaged as “private cloud” which aren't anything like cloud computing but focus on automation. These offerings don't add enhanced developer or user services. IT has to assemble them from open source or out-of-the-box tools. It requires adding security, log in, and configuration support.

Microsoft CEO Satya Nadella, in 2017, focused on the intelligence edge. Microsoft rolled at Azure Stack, a miniature of Azure's cloud, but it has a fraction of the services offered by Azure's cloud services. Amazon also began delivering Snowball Edge as an appliance and will probably begin pushing it harder soon.

Intelligent edge is not the same thing as a private cloud offering. Instead, though it gives users identical services as the public cloud, it's housed and accessed locally. Just like your cable operates off a set-top box, and is managed from a centralized cloud source. What does this mean for the near future?

In 2018, traditional private cloud markets will dwindle, and momentum will pick up for intelligent edge. New companies will scoop up these repackaged or enhanced offerings. Integrate offerings, or specific vertical apps will be employed to fulfill growing demand.

AI Raw Tech and Embedded Features    

The quick rise of machine learning in 2017 caused an enormous amount of hype, as did AI advances. They are mainly used by big data and market leaders in the web: Google, Amazon, and Facebook. AI is important to most enterprises, but it's not time for the average organization to search madly for scarcely available data scientists - yet. Many organizations are rushing to staff up to create AI models that fit their operational needs from scratch.

Salesforce built an AI into its offering, which leveraged the huge customer data components of its applications. Other companies are mimicking this in an attempt to embed AI as a feature in their offerings. There's a vertical focus on AI, and it's likely we'll see solutions pop up for verticals or targeted industries. So, if you're looking for new retail, healthcare, security or other specialized cloud apps, they're coming soon. Best of all, you won't need to untangle regression algorithms to manage these solutions. Instead, they give you data and parameters and include an AI model for your applications.

AI has been buzzed about for a decade, but this new field still has overlapping products with little standardization. If you use a framework like Spark, TensorFlow, Python, and H2O, you have to use the same platform for the scoring. In 2018, AI models will be defined, open and made compatible across platforms. As the technology matures and developers catch up, AI will become easier to use without giving up the many configuration choices available to customize your enterprise applications. Additionally, more solutions will arise in the realm of automation. This means that applications for building, training, and deploying your AI will crop up to maximize and expand the way your AI works for you. As an example, new AWS Sage Maker has many of these features already.

From Big Data to Continuous Data

In recent years, companies have developed big data practices with their IT functions centralized. These systems gather, safeguard and analyze business data as well as logs for future apps. Data is placed in data warehouses or Hadoop clusters. It's then used by data scientists to execute batches and create reports or dashboards. Leading analysts agree that this model doesn't work anymore.

Up to 70 percent of companies receive no ROI at the expense of managing their data this way. Data that cannot be leveraged into actionable benefits isn't very valuable or insightful. Integration is the first step to get the data into a format that can be consumed and used by the business. Otherwise, fresh data just joins the compost heap with no real benefits. This happens in Google and Facebook recommendations as well as ignored ads.

Data insights are a necessary component of effective business applications. For instance, a customer goes to a website or engages a chatbot and needs an immediate response based on their recent activities. Sensor data from IoT or mobile devices is continuous. It requires instant actions. Security monitoring, predictive maintenance, and other responses are what make these apps useful.

In this process, there is visual data that has to be inspected on the fly. It's a matter of import for national security and surveillance. Retailers also rely on it to get point-of-sales data. It gives insights into customer preferences, inventory status, and instant recommendations.

The Bottom Line

Technologies like Hadoop and the practice of data warehousing are already ten years old, before the age of AI, in-memory, stream processing, and flash. Companies are realizing that they provide little value in creating data lakes required for AI to function optimally. So, there has to be a shift in purpose and in the way data is collected and held. In order to perform data mining, a conversion to simpler cloud technologies is necessary. 

In 2018, we are experiencing a shift from big data to quick data and continuous data. Data has to move continuously to be available for consumption immediately by any number of sources for internal and external applications. This data will be context-rich, instead of flat detail with no added value. Real-time aggregation and pre-learned or continuous learning models drive AI, which generates an immediate response to end users. This happens in real-time and, more importantly, interactive dashboards.

Developers can take advantage of prepackaged cloud offerings, or they can choose to integrate their solutions via cloud-native services. At the company level, the focus has to be moved from IT operations to the business. Data-driven decisions need to be a built-in standard feature for all apps that impact business logic and customer interactions.

Whatever you decide to focus on, you will need the resources to execute your plan. Call ICS to find the talent you need to make it all happen. Click below to get started right now! 

Find Talent NOW