If Operational Intelligence Can Make a Teenager More Efficient- Imagine What it Can do for a business?

Achieving operational efficiency through the use of real time data can help a business become far more profitable and make smarter decisions.   We see the new information come at us about this  every day in the IBM CFO study, the CMO study and the recent study with MIT.



Data and information makes us smarter.

So being a person who loves data, I got to thinking.  Can data make my teenager more efficient? Can I leverage real time data and tools to achieve the impossible?

The answer is yes.

As with any parent of a teenager, we struggle to teach them important things like time management and responsibility as they grow.  I also get frustrated that she never seems to have time for thing that I ask her to do?  “Please clean your room today.” reply- “Mother, I don’t have time, I have way too much to do”.

I started to think about this.  How can a 14 year old not have a spare 15 minutes?  How do I make my teenager more operationally efficient?

I had a theory and I had a goal.  My theory was that my soon to be 14 year old daughter was spending way too much time texting, which was distracting her from completing tasks like homework quickly.  I say this was a theory because I didn’t have the data to back myself up.  My goal was to to help her be more efficient with her time. Yes, I will win this arguement with a little data.

Off I went in search of data for two reasons, one is to validate my theory (my teen spends ALOT of time texting) and the other was to simply just to let the data speak to me.

So what did I find out?

My daughter sent and received over 18,000 text messages last month (yup I was right that is ALOT of texting)

Those messages ranged from 6AM until 1AM (oooh – discovery!)

Now here is the key part, putting that data in CONTEXT in order to reach my goal of a more efficient teenager…

1) 18,000 text messages take approximately 30 hours of her time per month

2) If my teen sleeps 5 hours a night, that leaves approximately 126 waking hours per week

3) 30 hours averaged out daily into waking time means that she spends 9.5% of her waking hours texting

30 hours a month, 9.5% of her waking time- oh wow guess who will never lose that room cleaning argument again?

Ahh yes, I love data!!!!

My theory is validated and I discovered that my daughter can text in her sleep (ok maybe not).   So I take the right steps towards operational efficiency.  I restrict texting hours on her phone using the tools on my carrier’s website and I now have freed up several hours of my daugther’s life!

Imagine, being able to use data real time and being able to put that data into context that increases efficiency, profit and competitiveness (the one with the data will be the smarter competitor).

This is operational intelligence, having the information you need when you need it.

Key learning- the person using the data needs to be able to put the data in context in order for it to be truly useful, truly powerful

So it’s not just about technology, without putting the data in context, it is far less powerful.   That said, you need a solution you an deploy faster and easy to use so the smart people can easily get the information they need to make smarter decisions.

If you went to IOD this year, you may have seen people walking around with iPads with Mobile Cognos running on a Smart Analytics 5710.   This solution can help you achieve operational intelligence starting at $50k , including Cognos.


The Netezza 1000 can be up and running in 24 hours, thats 24 hours closer to answers you never had before!


What are you waiting for?

You too can be the “mad parent data scientist” at home, its the parental contribution to building a Smarter Planet.

Then at work, build a smarter business with Smart Data Warehousing Solutions from IBM.

Let’s build a smarter business (and smarter teenagers!)


It’s All About the Analytics

One of the greatest things about IBM’s new Smart Consolidation strategy is that it’s helping us get back to focusing on the important things when it comes to data warehousing—hey it’s all about the analytics.

Come on, lets face it. For years we have dug ourselves into the gory details of architecture, workload management, monitoring. I know personally I have spent the last few years discussing how to build out the architecture half the time and the other half how to manage them. What I really miss talking about is the reason we build these things in the fist place- yup it was the analytics.

It’s been so refreshing to have conversations about customer insight, proactive campaigns, deeper insight for fraud, trend analysis,…. this is the fun stuff.

Ten years ago many of us were talking a lot about CRM- remember? Some of you may even remember multi- channel marketing or horizontal marketing. Maybe we were ahead of our time back then but with the data and technology like our smart phones available today, these goals can indeed be realized like never before. Back then our warehouses were taking data in batch, now we can stream in and analyze petabytes of data at a time. If there is a buzz about a hot product, a retailer can capitalize on that by pushing a proactive coupon to their customers and track the success of that campaign so they only get smarter.

CRM topics are making a comeback for sure. As businesses we have to be smarter about who our profitable customers are, what they are buying and how we get them to buy more and stay loyal. This is not just a retail thing, nope. Banks are offering more products and services through more channels than ever before. I don’t know about you but I consider my smart phone yet another channel that my bank uses to communicate with me. Think of the possibilities!

Oh yah- this is why I chose a career in business intelligence.

So it’s time to shift our focus, less on architecture and more on enabling true business value.

The first phase of Smart Consolidation is a direct focus on optimizing your ability to deliver and manage ongoing analytic capability in your architecture. The key goal here is to reduce your time to value on analytic applications significantly. With Netezza solutions, you can be up and running in 24 hours. In addition, you will cut your administration and management of these applications down to almost nothing. This lets your team focus on delivering even more value to your business.

Yup, it’s all about the analytics.

IBM’s new way forward in Data Warehousing

Sometimes you have to stop and think about where you came from to get a good direction forward.  I think we learned a lot in our journey to consolidate our data marts.     Heads up, the dynamic environment of our business is pushing us towards a new strategy.

Consolidate based on best practices (not just everything)

Let’s be realistic, maybe we don’t consolidate everything into one big structure.  Instead, let’s apply certain principles or methodologies to what we do consolidate.  Allow offloads of the Enterprise Data Warehouse (EDW) where it makes sense- especially for analytical applications that the business wants quickly.

I know this is the opposite of what Oracle and Teradata are telling you but it’s time to call it out— those big monolithic structures did not answer all our prayers in data warehousing. The overall success rates of EDW projects were incredibly low.  Why?  It’s really a combination of creating too much complexity and the failure to deliver new value in a timely manner.

IBM’s vision for the data warehouse architecture going forward is based on the concept of Smart Consolidation and leveraging analytical appliances within an EDW Ecosystem that is fully optimized for delivery of analytics.

What is Smart Consolidation? 

We can consolidate smarter by matching the right compute capabilities within the Data Warehouse ecosystem to the workload where it’s most optimized.

This is different from consolidate everything.  In our rush to consolidate, we lost the hearts and minds of business because we created a demand for the data and then couldn’t respond quickly enough.

Let me give you a real life example.  We had a client who had built a fairly complex EDW system.  Over time, they started to be challenged with query performance and the client was quite anxious to roll out some new applications in sales and marketing.  Surprise surprise, the business did not want to wait.  Rather than frustrate the business with an increased timeline, this client chose to offload to a Netezza appliance.  The result?  Queries ran 24 times faster and they were able to achieve lower TCO.    When it comes to the EDW, we all want to do a few glory marches a year with our business users- come on admit it!

Think about this….

When you go to Google to look for information, would you accept an answer of “we’ll get back to you in 3 months?” 

Ok, I didn’t think so.  As IT professionals we have to focus on being responsive to our business needs. Our business users have the “google mindset” – they just aren’t going to wait for the answers they want.  We need to increase our agility in data warehousing.

So consider this- the Enterprise Data Warehouse (EDW) is now part of an intelligent business ecosystem.  This ecosystem offers all the capability you need to deliver business intelligence and maintain governance overall.  It allows you to match the work within the ecosystem with the right computing solution AND it helps maintain performance across workloads.  So whether you need to analyze unstructured data, leverage a queryable archive or increase the efficiency of some targets analytics – we can help.  This is the evolution I am talking about!

You are probably wondering HOW we would do this?  The answer as any good Data Warehouse Architect knows is in the overall design of data architecture.  What we are talking about is a smarter way at accomplishing the data integration and delivery for the enterprise.   Our brilliant counterparts on the IBM Global Services Business Analytics Organization (IBM Global Services) team have come up with a new way to tackle this issue today, with a new generation design.

This design features a “data synergy hub” that allows us to maintain lineage of the data as well as an atomic layer or single source of fact- but also take advantage of appliances where it makes sense.  Of course everyone always wants that black and white answer of where does it make sense and I am going to repeat something very important.  Your business requirements need to drive your design, not the technology.  If you need some help being business requirements driven, call the GBS team, they can help contribute to your EDW success.

This new way of thinking in regards to both design and the infrastructure allows us to  leverage and deliver more sophisticated analytics within the ecosystem around your data warehouseYes, you read that right- we are evolving thinking from a single data warehouse architecture to an ecosystem around the EDW that allows us to have consolidated and governed enterprise data but allow offloading to other systems that can help the goals of delivery and performance, especially for analytics. 

It’s brilliant, it’s beautiful, and it’s the answer we have been waiting for.

Be a part of a “Smarter” Data Warehousing Revolution

Demands in our business today require more agility and more focus on time to value.  The “old way” of creating a large monolithic structure really complicated our ability to deliver.  We started to focus too much on managing performance as opposed to delivering value.  So continue down the path of consolidation for the right reasons.  Aim for that single source of fact but accept that the answer may be to leverage new appliances and technology in your ecosystem to accelerate delivery.

Think about how you articulate this vision to your company.   New tools like the IBM InfoSphere BluePrint Director have been built for that very reason.  Now the Information Architect no longer has to guard his whiteboard masterpiece from the night cleaning crew.   That vision can be better articulated and tracked through the progress of the project. Get those drawings off the whiteboard and into a tool where you can track progress, requirements and design changes.

Wow things have come along way. What are you waiting for?  Join us!!

Learn more about IBM’s Data Warehouse Appliances

Neteeza – http://www.netezza.com/data-warehouse-appliance-products/index.aspx?

PureScale – http://public.dhe.ibm.com/software/data/sw-library/demos/purescale/purescale.html

Smart Analytics – http://www-01.ibm.com/software/data/infosphere/smart-analytics-system/

InfoSpherenBlueprint Director – http://www-01.ibm.com/software/data/infosphere/foundation-tools/

What have we learned from data marts and consolidation?

Why we built data marts in the 90s

Business intelligence at IBM started with data marts, which helped solve the problem of reporting and analysis for specific areas of the business- like financial or marketing. However, we also found they created problems, like the ability to deliver compliance, security, governance and the ever popular and dreaded data redundancy! When CFOs started going to jail for not having their facts straight, we had to get serious about compliance and governance.

Evolving from Data Marts to the Enterprise Data Warehouse

IBM and its competitors began to focus on building Enterprise Data Warehouses (EDW) to solve the problems of data marts and data sprawl. We built consolidated and monolithic infrastructures that were really good at being that “single source of truth” we were aiming for. And as we pulled all the marts into the EDW, the concept of virtual marts came along with new technology that allowed us to have cubing-like capability in the engine of the database.

We wanted control over the data so we tried consolidating, but maintaining control caused other problems to appear. It was far more difficult to maintain performance in these systems, especially for some analytical workloads. Smaller vendors in the market place were building cheaper appliance-like solutions to go after the analytics workloads and offload that functionality from the EDW. The appliances really appealed to the line of business who wanted things FAST and cheap and that demand created a big surge of appliances in the marketplace starting in about 2003.

But Data Marts kept popping up! Like playing “Whack a Mole”

Ten years later, and after all the things we’ve learned, we’re once again seeing this trend of marts popping up. Have we figured out a way to solve the original problems of marts? Well kinda, but we made some tradeoffs along the way. One of those tradeoffs was having control over the data but we found the EDW wasn’t catering to business needs, like ROI and fast time to value. I have seen ROI numbers as high as 145% – no kidding. If you were a CFO, would you choose fantastic ROIs or the EDW?

Why CFO’s keep choosing data marts

Ok, Ok so I know marts happen but it’s WHY they happen that keeps me up at night. How is it that we built these wonderful EDW structures but still somehow failed our business counterparts? How could they not care about governance and compliance? And if we don’t manage the deployment of these marts, we’ll end up back where we started a twenty years ago.

Not to be forgotten though, we can now declare data warehousing a success because there is more pressure than ever to deliver more of what the business wants. Quite the conundrum eh? How do we manage the trade offs of data governance and the ability to respond to the needs of the business?

A Smarter way forward – learning from both data marts and the EDW

I don’t think the answer lies in simply saying that throwing everything on one box will get you to where you need to be. Sure it’s simple, sure it sounds good – in theory. Oracle is still pushing a strategy of one size fits all – what that tells me is their maturity level in data warehousing is about IBM circa 2003 – but hey that’s ok they will catch up.

In reality the monolithic architectures have fixed some issues and created others. We fixed some governance, metadata, security and compliance issues. But we created such complexity in our modeling and methods that we aren’t as useful to our business sponsors.

In a nutshell, we kept our architects and DBA teams so busy they lost their connection to the business. I think you have to be far more practical about how to approach data warehousing.

The answer to what lies ahead is being able to take advantage of consolidation and also to be able to meet time-to-value, in particular for analytical applications. We must match the needs of the business to the right computing technology.

IBM’s evolving business strategy for the EDW

In my next post I’ll look at IBM’s new business strategy that has evolved from what we’ve learned from data marts, the EDW, and 20 years of experience in business intelligence.

Why Won’t Oracle Come out and Play?


Oracle is so very confident in the performance of their Exadata solution — but then why do they shy away from a good ole fashion on-site proof of concept (POC) ?


We know choosing a data warehouse system is a big decision and we understand the challenges of running a proof of concept.  We know you are betting your job on the decision and we know you want to see performance in the walls of your organization.

Smarter Question:

If Oracle is so confident why do they make it so hard for customers to evaluate their solution on site? Hmmmmmmm

Last I heard the customers is always right

Ok so not so long ago in a not so far away place, the POC/benchmark team reported to me for data warehousing.  As a part of my initiation, I learned of the joys and pains of running a customer POC, as a vendor.  I mean let’s face it, it was a character building experience for me.  I would have LOVED to put my hands on my hips and tell customers that “if you want to try my solution YOU have to come to me in my nice comfy and completely controlled environment”!

But then cold hard reality set in and I remembered, the customer is always right and 9 out of 10 times, the customers wanted their proof of concept on-site.    You see, I realized that it was just as much of a pain for them as it was for us, they had to get the people, the power and floor space and in some cases, they pulled all nighters too.  Why did they do all this?  Because the decision to buy a data warehouse solution is just that important.

Why would you accept no?

It’s just simply baffling. I mean in my “sales dreams” I ask a customer to make an IBM decision and they just smile and say “absolutely”.   But I know we have to earn that decision.  I know we have to earn that trust and above all I know all too well, we have to prove ourselves in a proof of concept.   So why would anyone invest millions of dollars in a system that they can’t test drive in their own facility?

Oh wait, there IS a 3D demo on Exadata @ Oracle.com, yah I would totally bet my career on a 3D demo (NOT).  Is it they don’t have enough machines? Are they worried it will fall off the back of a truck (ok I have a story on that for another time), do they have less confidence when the machines is out of their control?   Is Exadata secretly a huge pain to get up and running? Does it require an army of Oracle-ites? (link to Netezza movie #3)  Why do they get to be different?  Why does Oracle get to say no?

The Analysts tell customers to run a proof of concept

If you ask an analyst or any major industry consultant about making a data warehouse decision, they will tell you to run a proof of concept.  Curt Monash too.  In fact there is no shortage of advice, help and even methodologies.  It’s just how these decisions get done.  IBM gets it, Netezza gets it too. In fact they have made taking the TwinFin out for a test drive as easy as filling out a short form online.  No kidding, it’s that easy.  (Take the “Netezza Testdrive“).

So Why Does this make Oracle so cranky?

That’s the question of the day isn’t it? If IBM & Netezza can mobilize for a customer POC then why can’t Oracle?  This got me thinking.  Yup it’s true we have a lot less control over the test when its not in the walls of our lab-  but isn’t that exactly what the customer wants to see?  They want to know just how hard this is going to be.  They want to truly understan from hitting the dock to query performance how much work this is going to be. They want to know if they will have weekends and holidays off for the next 2 years.

If you were betting your job on it- wouldn’t you want know that?

Let’s Start a Revolution

I say we start a revolution and demand systems to be up in running in 24 hours, NOT 24 days. Let’s raise the bar, give customers what they need and want to make this important decision.  I say if we offer on-site customer proof of concepts, well so should everyone else who is serious about this business.

So if you can’t take the heat Oracle, well then you stay in your comfy air conditioned labs and we will be happy to show your customers our solutions on site.

Is Oracle Exadata REALLY a Data Warehouse Appliance??

Oracle: Oracle claims their Exadata solution is a one-size-fits-all Data Warehouse appliance (source: Oracle Exata)

IBM: We don’t believe anyone can combine every function in the same box for optimal performance.  An appliance is by its nature single-purpose.

Smarter Question: Is Oracle Exadata REALLY a Data Warehouse Appliance??

Well lets start with how we define data warehouse appliances.  From the all knowing source of Wikipedia…..

In computing, a data warehouse appliance consists of an integrated set of servers, storage, operating system(s), DBMS and software specifically pre-installed and pre-optimized for data warehousing (DW). Alternatively, the term can also apply to similar software-only systems — purportedly very easy to install on specific recommended hardware configurations or preconfigured as a complete system – a true appliance.

DW appliances provide solutions for the mid-to-large volume data warehouse market, offering low-cost performance most commonly on data volumes in the terabyte to petabyte range.

Netezza shakes up data warehousing

Ok so let’s get real, those of us who go back to the “dark ages” of data warehousing never used the word “appliance” until this little company called Netezza came along one day and shook up the market place with the first data warehouse appliance.  Netezza’s had the ability to provide the fastest time to value and were nimble enough to do Proof Of Concepts at the drop of a pin.  In fact, Netezza still offers the ability to “test drive” the solution, without all the hassle- just go to Test Drive TwinFin if you want to know more. Customers loved this, not only could they evaluate things faster but the whole idea of not having to do all that pesky integration work was really inviting.  In response to this disruption in the market, many vendors like ourselves at IBM realized that we had to do things differently.  We had to make our data warehouse solutions more “consumable” and it just so happened we had hardware, software and services- how totally convenient.

IBM brings out its first DW appliance

In 2004 IBM introduced the first generation of data warehouse appliance called the “BCU”.  Today the Smart Analytics is the 5th generation of IBM heritage hardware and we were recently lucky enough to add the “original DW appliance” to the family- Netezza.  Other vendors at the time like Oracle, Microsoft and even HP- well they were not as lucky to have all the parts and pieces they needed and found themselves quickly looking for a “serious commitment”.

Oracle finds appliance love… again.

HP and Oracle’s marriage only lasted a year, we are not sure what happened but we are pretty sure the marriage was annulled because neither one will talk about it.  Oracle decided to marry Sun and put a few billion behind that commitment. Microsoft married a small firm called DataAllegro and has recently been cheating on the side with HP as well.   Teradata felt the whole team “appliance” was below them for years.  That is until recently when they saw their market share being chipped away and suddenly they are in the appliance business with a new line of solutions.   All of these unions are all really in the quest to become an “appliance” because as you can see from the above definition you need a complete integrated system.

Exadata is not a data warehouse appliance

Ok let’s get back to the question.  Is Oracle Exadata really a data warehouse appliance?   So I am going to just come right out and say no.  Why? Well first of all, Oracle claims Exadata will do everything from data warehousing to OLTP. Hmmm hey now, we were talking about an appliance here, something built and optimized for a single purpose.  Everybody in the market has a focused and optimized solution for data warehousing BUT Oracle.   When I buy a toaster, I don’t really expect it to also provide other services but it should be easy to use, toast well and work when I press the lever.  If my toaster also does other things like say deep fry – well can I really still call it a toaster?  Similarly, if Exadata claims they can do it all, then it isn’t really an appliance.  One last point, don’t ask Oracle to take a test drive of their “appliance” at your company’s site.  I hear this makes them very crabby

Appliances should be easily supported

The other important part of being an appliance is how the solution is supported.  The customer should only have to make one call to get help. Administration and manageability should be easy and the total cost of ownership should be low.  So rumor has it that there have been some bumps in the road with Exadata on things like software updates.  At Oracle Openworld, David Moore from BioWare stated that it’s both very important to keep up with patches and that they occur frequently. Hmmmm. There are numerous public references to patches causing all sorts of challenges for customers, in particular 4 & 5.  This doesn’t sound easy to manage.

The road less traveled

It sure seems like Oracle looked at the crowd gathering around data warehouse appliances and they chose the road less traveled.  Question I have is why are they the only ones going down that road?

Join the conversation at SmarterQuestions.org

Learn more about IBM’s Data Warehouse Appliances



Smart Analytics