Automated data management software delivers faster, more accurate financial reporting.
The latest from our blog
Integrate everything into your Tableau dashboards at a fraction of the time and cost of Alteryx
JOIN US AT #TC18
Meet our management team and contact our offices in the UK, USA, and Australia
Opportunities for VARs, ISVs and software vendors
Keep-up with the latest goings-on at ZAP and the events we’re attending
PLUS five-point plan for automated data governance
Raise a support ticket and explore our knowledge base, user forum and more
A wealth of materials to help you become a data driven business. Videos, webinars, mentoring and more
Get in touch
September 7, 2018
By Trey Johnson
No comments yet
Tech evangelism is more about listening than talking. Having recently presented at Azure Data Fest in Atlanta – on the topic of “BI Workloads and How They Change in the Cloud” – this latest Tech Notes blog is less about my PowerPoint slides and much about what the audience had to say.
I met around 50 people at the session – one-third of the attendees at the event – and they were an audience hungry for education, so I asked some leading questions…
The reality is likely most people working with data are broadening or seeing their roles broaden to include BI/DW/Data Science.
The Modern Data Platform
I used one graphic to portray many of the capabilities of the more modern BI/DW/Data Science platform and it just happened to be from Microsoft:
The session spent time using the above as a means of talking to the differences between “On Premises” and “Cloud.”
Collecting and Managing Data – On Premises
I delved into the realm of “Collect and Manage” on the Data Platform wheel Microsoft had in their graphic. Specifically this:
After sharing my view of the traditional “On Premises” architectures typically supporting only the non-whited out boxes below…
…I asked “What are your Data Sources and where do they end up today?”
The group confirmed their architectures were predominantly fed by relational data sources coming from applications and agreed with my assertion that the data sources were generally relational (or structured). Pressed a bit further most offered there were Data Marts and/or Data Warehouses. The crowd was still warming up and I had assigned them to teams, BLUE or RED, where each response earned a point for the responding team.
I found some people starting to verbalize concern about moving their Data Warehouse to the Cloud while a small group (probably two from the same company) had taken bigger steps with the cloud (Azure specifically) and were happy to report positive progress!
Transforming and Analyzing Data – On Premises
As we made our way around the Microsoft Data Platform wheel, we discussed “Transform and Analyze”. This is a fairly heavy-duty part of ANY Business Intelligence/Data Warehouse/Data Science platform as the real value starts to materialize here. Information is being transformed, prepared and modeled for inclusion in data stores, which support Analysis.
We talked a fair amount about the below elements which are support “On Premises” and the whited out items again being more “likely” in Cloud environments:
Most in the room agreed with there being variances depending on if your approach to loading the data used “ETL” or “ELT”
ETL or Extract, Transform and Load – is a process by which data is collected from a data source and transformed in flight on its way to a Data Mart, Data Warehouse or other decision supporting data store.
ELT or Extract, Load and Transform – is a process by which data is collected from a data source and loaded initially in its originating form. The originating format of this “staged” data is then transformed after it has landed in the BI/DW environment.
There was also agreement on data, especially “On Premises”, being modeled in a more traditional Dimensional Model, popularized by the Kimball Group.
What makes the concept of Transform and Analyze that much heavier architecturally is not only is there a level of relational preparation of the data but there are potentially other modeling activities for structuring the data for Analysis. In Microsoft’s world, this involves Analysis Services, which was originally offered years ago and has been refined on the SQL Server platform. Today, Analysis Services is ALSO available as an in-memory offering on Azure (and it is the technology which underlies PowerBI’s in-memory data modeling) called Tabular Models.
Interestingly, I asked Team BLUE and Team Red to share their perspective on Analysis Services and the response to “Is Analysis Services in use or do you simply provide data relationally from the Data Warehouse or Data Mart?”
The answers were roughly 15-20% on Analysis Services (in some form) and the remainder said they either didn’t use it or didn’t use it and relied on Power BI to provide the modeling (which means they were still using tabular technology, just in a limited fashion)
Interestingly, there was a much LARGER dependency on Microsoft’s Integration Services technology (SSIS). I’ve spent literally two decades with this technology (including DTS) and I imagine this is one of the big anchor points for most when moving from On-Premises to the Cloud. Microsoft has a technology called Azure Data Factory, which we’ll go into further detail a bit later on.
The session I delivered wasn’t the time to be wearing a ZAP Data Hub hat and talking about our products but what I will also discuss in a bit more detail, later, is the simple reality that ZAP’s Data Hub seriously speeds up and makes easier the processes around “Collecting and Managing” and “Transforming and Analyzing” the data, weather information or the Data Platform be On Premises, In the Cloud or a Hybrid of the two.
Visualizing and Deciding – On Premises
As we completed the circle of the Microsoft Data Platform Wheel, we reached the “Visualize and Decide” section.
We spent a lesser amount of time speaking to the elements of this area of the Data Platform.
This was due to the fact most of the audience agreed with my assertion that On Premises or in the Cloud, much of the visualization activities are still supported when you move between the two.
The IT Workloads being Managed on Premises
We’ll conclude this Blog with a view of the Workloads being managed on Premises.
The group agreed with all of the above and we focused on the Pro’s and Con’s of having a BI/DW/Data Science Platform On Premises.
Pros and Cons of the Data Platform – On Premises
Some of the Pros discussed included:
Some of the Cons the group agreed with and raised included:
My final questions to Teams BLUE and RED were:
Is it an absence of “Pros” or the presence of “Cons” which are driving you to look to Azure?
The consensus was they were wanting to really understand and leverage the opportunity of the Cloud technologies and innovations. Some were focused more so on specific CCloud-orientedtechnology but all seemed to have a level of interest (which is good considering the event was called “Azure Data Fest”!)
Do you have other concerns about moving to Azure?
There really weren’t many objections but honestly, I wasn’t expecting much in this environment. At ZAP, we’ve heard people concerned about unforeseen costs and risks but it so rarely comes through with a highly technical audience.
So, that’s where we will start our post next time, finishing off the recap of the session and talk about all the great interactions around the Azure technologies, which make up the Microsoft Data Platform. Who knows, maybe you have a vision of your Data Warehouse looking like the diagram below?
Stay tuned and, until then, I mentioned at the start I love hearing from people! Feel free to give me a shout/follow on Twitter @TreyJohnson.
Trey Johnson is ZAP’s Chief Evangelist. Based out of Jacksonville, Florida, he brings experience from leading various boutique BI software and national consulting companies. A published author, speaker, and consultant, Trey sat on the PASS Board of Directors over multiple terms, concluding as their Executive Vice President. He was a long-term member of Microsoft’s BI Partner Advisory Council and has spent the last 25 years delivering business intelligence, data warehousing, and data management solutions to businesses of all shapes, sizes and “data challenges.” Follow Trey on Twitter and LinkedIn.
Comments are closed.
All Rights Reserved