If you think you have seen everything in the BI space, I’m sorry, but you may be wrong. Traditional BI solutions for large enterprises consider the data warehouse to be the basis of BI solutions. QlikTech and PRELYTIS, two innovative software providers I recently contacted, could be changing this paradigm. What follows are some of the insights and interesting proposals I gleaned from our conversations…
In traditional BI software solutions, the data warehouse plays a fundamental role as the central data repository for all the data gathered from different systems. This repository has the required structure for analytic purposes, and prepares the data to be used as the basis for intensive and massive analysis processes. Some vendors, however, are agents of change in this regard, in the sense that they are taking new approaches to handling and storing data: while they still extract data from the source systems, they avoid the use of a traditional data warehouse structure, favoring instead an analytic data structure model to speed information analysis and the data visualization process. Recently I had contact with two of these vendors in the BI space: PRELYTIS and QlikTech. These vendors might be changing the way we design data warehouses, and may even render them obsolete.
PRELYTIS: The LiveDashBoard Solution
LiveDashBoard is a BI software solution offered by PRELYTIS, a vendor headquartered in Paris (France). With customers across several types of industries (insurance and finance, health care, energy, and distribution, among others), LiveDashBoard has a solid number of users, mostly from Europe and Asia, from large companies such as Barclays, Total Petrochemicals, and Groupe Kapa Santé, to name a few from its client list.
LiveDashBoard bases its success on an innovative way of developing BI dashboards. LiveDashBoard enables users to develop extensively interactive dashboards to interact and collaborate with other users. Data is gathered from widely diverse sources into data dictionaries within LiveDashBoard, at which point all data is merged to created data catalogs. Without any script, it’s possible for users to store data from all sources, store the data in a proprietary analytic database, and start the analysis and data visualization process, avoiding the complex development process of building a data warehouse.
Nothing is installed on work stations or laptops: with a single Web-based architecture, all users (from the server administrator to the data consumer) can access LiveDashBoard via a Web browser. This enables users to access all of the product’s functionality, no matter what operating system or platform they are using. LiveDashBoard also counts on a strong BI core to deliver information the way the user wants:
• Alerts and conditional reporting
• Business metrics and balanced scorecards
• Mobile access
This variety of information vectors comprises just a portion of the functionality features that LiveDashBoard can provide to users. LiveDashBoard also has a set of business performance management functions in place, which enables it to provide functionality that is complementary to its core BI functionality. Platform-independent due to its Java-Ajax design, LiveDashBoard can be deployed on different types of operating systems, which enables Prelytis to provide a solution to a vast number of organizations of differing types.
QlikTech: The QlikView Solution
QlikTech was founded in 1993 in Lund (Sweden). QlikTech develops and distributes QlikView, a BI system based on a multi-dimensional database, with an innovative patented in-memory technology. Based on its in-memory approach to BI, QlikView has gained increasing popularity among a large number and different types of customers around the world, including Campbell’s, Toyota and Panasonic.
QlikView’s innovation lies in its in-memory data storage and analysis technology. Information is gathered from multiple data sources and stored into memory; then, analysis takes place in-memory, which speeds up the analysis of large volumes of data (just seconds as opposed to minutes or hours). Along with its in-memory technology, QlikView offers a complete set of BI tools to extract and consolidate data from different sources, as well as a complete set of data reporting and data visualization tools. In the latest version, QlikView has incorporated a very interesting set of functionality for mobile devices, which enables QlikView to provide fast data analysis to a group of mobile users.
QlikView is designed to run on Windows platforms, and it supports almost all Web browsers for its client interfaces. Its server version can be installed on 32- or 64-bit Windows servers.
And This Is Just the Beginning…
Definitely, we are far from seeing the end of the story when it comes to the development of innovative BI tools. And with the incorporation of new technologies, the BI landscape will see newer and better solutions to fulfill the needs of information analysis.
You can take a minute to compare the functionality of these two BI products using the TEC Advisor.
The initiatives of these two innovative BI vendors raise the following question:
Future of Data Warehousing
The Warehouse is supported in customers by careers and massive sunk dollar cost - so many will remain for a long time because too many people would lose face by removing it.
The Warehouse is pushed heavily by several of the most powerful technology companies on the planet; Because the warehouse can only run on their RDBMS platforms, and their share price is dependent on the number of database licenses they sell.
For them, any move away from the warehouse to a new technology will cannibalize their core revenues.
“Elephants Dance with Elephants” and so the big customers will always buy from the big vendors (who MUST sell RDBMS) - for a whole host of reasons that are nothing to do with the inappropriateness of the technology or the massive costs involved. They buy from the big vendors often because of PERCEIVED risk (everyone else has bought it, so my job is safe if I buy it).
Eventually new technologies will replace the warehouse. But not until all the stake holders in the massive warehouse projects have moved on to places where their careers cannot be hurt by the change.
Don’t get too hung up on the technology - it is hardly ever the reason people buy.
Livedashboards and Qlik view have huge draw backs
Livedashboard can at the most be called as a web replacement of excel and may not be able to cater to enterprise level needs
Qlik view in memory again neglects ETL process.
Bad data in is bad BI out
also requires complete up gradation on hardware to 64 bit machines and huge RAM upgrades
I don’t see any of these are really having any radical technological enhancements
These BI tools really do look useful, however I don’t yet see how they make the DWH entirely obsolete.
The DWH does not just store data, but consolidates, normalizes, maintains history (that the source application might not do at all) as well. If you want to do meaningful analysis, you should have consistent data, so you need some kind of pre-processing and a store for the cleansed data. (except if you have a modern, fully on-line and synchronised architecture all the way, something I haven’t seen yet in any larger-scale enterprise).
As a Qlikview Reference I am a devotee of the tool and it is revolutionisng our business, but Peter is right the DWH has a - albeit small - role to play for the reasons described.
I guess it all depends on how you define “data warehouse” and where you use it. As a consultant to both E&C firms and O/O, I have to agree more with Peter. We use data warehouses to collect and cleanse data from one engineering application so that it can be shared with other. This helps us provide consistency and accuracy. It also provides the data history that we sometimes need. We also deal with data that is more rapidly changing and is not very static.
In the situations I deal with, engineering group “A” may enter data in their system but they are not ready to “release it” to the other groups yet. Status settings and “publishing” routines help us control that. Would something like that be available with these new systems? Maybe not, especially if the source system does not keep a history. How would these new systems find the last “issued” data if the user in the source application is working on a possible upgrade of a system in a plant or facility?
Now, I do think that these BI solutions have their place and sound great. I just am not convinced that they will do away with data warehouses.
data warehousing is a concept, and as such won’t be obsolete, at least any time soon. implementation of the concept, however, will change as technologies change. the need for standard data definitions, aggregation, and so on won’t go away, although every component in a DW implementation may take a different form over time. i do appreciate the need for a good sensational byline to attract readers ;)
Article about BI vs DWarehouse
We’re another BI software vendor who is helping to make data warehouses less necessary, with direct access to disparate data sources and pre-aggregration and data caching technologies.
But there will still be needs in large enterprises to use ETL technologies to cleanse and transform data and to use data warehouses to take care of the summarizations to deliver even higher performance.
So yes, new BI solutions can obviate data warehouses in some scenarios, but not kill them off everywhere.
Very educational. Thanks!
I wonder how the Gartner Quadrant for BI was built. PRELYTIS, the tool compared to QlikView in this report, is not included in the Gsrtner Report while QlikView was ranked in the top.
I can tell you that Gartner has a annual revenue requirement of $15M for consideration for the magic quadrant, so perhaps PRELYTIS is below that.
Yeah it’s not so long ago Qlikview wasn’t on there. I think the argument about data warehouses being dead is the wrong one to be having. Qlikview’s real benefit for me is time to value and agility - whether reading direct from data sources or a warehouse. Oh and cost and simplicity too, rather than trying to understand a crazy complex stack and what you need to buy you are delivering BI to your company after a few days consulting
We are also a vendor of a new style BI product, and it is possible to get the ETL, history, data cleansing benefits of the data warehouse plus the agility of development and end user simplicity offered by these new tools.
The best of both worlds at half the cost and half the effort.
Even Qliktech say that “In memory” is not their core differentiator, it is the structure of their data and cross referencing that is brilliant.
So if a new BI tool provided similar structures and cross referencing are available in large, scalable disk based data structures that perform well, then you have the same capabilities as DW plus the same benefits of QV.
That’s where we play, and it is definitely possible to combine the best of both worlds.
What this means also is that new generation BI like LiveDashBoard do not require a specific technology to be in place to deploy KPIs/business dashboards.
Companies’ IT infrastructures are constantly changing, moving away from traditional Windows + datamart/warehouse in place; new technologies come in place: open source, warehouses, and companies are expecting to pay less for the same. IT Heads and (budget holders) do not want to be stuck with one technology, costing always more. They are requesting for flexible solutions/inter-operable able to integrate with their current technologies, and even able to take into account their future technology investments.
I am not a specialist of QlikView but I have seen (press, news, national BI forums) that LiveDashBoard is being chosen over traditional BI (published by the Gartner) simply because it was working, because it required no or less development/scripting, leading to a lower TCO, and foremost because business users would get their KPIs, & business dashboards in a few weeks only, which, we shouldn’t forget, is the purpose of such a project.
Should we be listening to the needs of the market, or the marketing campaigns of traditional vendors?
[…] in this space. Recently, I had chance to read a very interesting write up made by TEC – Are Data Warehouses as Dead as the Dodo?, which is exploring a promising future of new BI technologies to replace data warehousing […]
Marion, the cynic in me says that until IT managers are motivated (paid?) to listen to the market, they will instead listen to the major vendors.
There is less risk to their jobs by authorising a $50Million SAP project that fails than there is in spending $50K on a PoC for a “Non Mainstream” product - whether it fails or succeeeds.
In fact the more they spend the less likely it is that anyone will ever say it failed.
Tools like QV and the other new products need to be business driven in order to break the stranglehold major vendors have on the IT manager’s safety net.
30 years ago Microsoft was that minor vendor, breaking IBM’s hold on the data centre & selling Office function to the business. Where are they now?
Many businesses now are again held back by their IT departments and their commitments to a few major vendors.
Time for a shake up - and it should come from the business.
[…] in this space. Recently, I had a chance to read a very interesting write up made by TEC – Are Data Warehouses as Dead as the Dodo?, which is exploring a promising future of new BI technologies to replace data warehousing […]
I’m a Qlikview customer and do not think it removes the need for a DW - see all comments above. Mainly because one benefit of Qlikview “in memory” is also a draw back if you have warehouses with Terabytes or even Petabytes of data. I think the technological game changer in the storage of cleansed, validated and history presserving data warehouse will be the emergence of Column based RDBMS with the improved IO profile they give over traditional row based RDBMS.
Qliktech and other analytical front ends can not retain all of the enterprise data. As long as the analysis uses the raw data that was initially loaded into them, they are great. If it is not there then a new process must be built to incorporate the missing data. If the selected data gets too large then performance suffers to the point where the system fails to meet business needs.
A system like Prelytis is really just the old Federated query system with a nice front end to simplify the use of multiple data sources. However,
1.It can not address the performance issue where the response to a query is determined by the slowest part of the federated system
2. It can do nothing about variable data quality across multiple sources
3. It can do nothing about rationalization of data across multiple sources