We just wrapped up a great trip to Spark Summit East. As I made my way back to the office, I thought I would share a few of the important lessons the team took away from the sessions and case studies presented by data innovators from Bloomberg and Capital One.

Companies of all sizes attended this year’s event, but no matter how big their big data ecosystems, a few themes were common across the board:

  • Businesses want answers and they want them now. This is no surprise considering Spark was created for streaming data. Accelerating mean time to decisions is a top priority for data analysis teams and the people responsible for building out analytics infrastructures. Tweaks to Spark and new software applications are popping up left and right to try to achieve this, but the folks we talked to were still not getting the performance they needed.
  • Data organizations need to make big data small. To be honest, this was one problem we weren’t expecting to hear much about at this event but were excited to be surprised. It’s no shock that Internet of Things, sales, social data and more are often creating more data than companies can easily handle or that they need. And now, data organizations are looking for ways to easily thin their data sets and make insights actionable in order to solve business problems. There hasn’t been a turnkey solution to date for intelligently thinning data at the edge, so we’ve had some great conversations about how the Ryft ONE can help.
  • Performance is still king in analytics, but ease of use is next in line. Whether it’s a 5- or 5,000-node cluster, everyone from developers to data scientists is looking to take an easier route. Finding a way to keep up with growing data sets, higher velocity data and increasing demands from business decision makers means that while real-time insights are necessary, finding a way to get answers without creating hardware and software complexity is another concern they’re looking to alleviate—often to no avail.

The Spark community remains a great representation of what big data users overall are looking for, and we had a great time in New York chatting with and learning from them.

If you didn’t have a chance to chat with us at Spark Summit East but want to learn about how Ryft can help accelerate insights while streamlining your data infrastructure, reach out and let us know that you’d like to schedule a demo.

Leave a Reply

Your email address will not be published. Required fields are marked *