Analytics and Business Intelligence – Introduction

The data you need to manage a business falls into two broad categories: data you know you don’t know, and data you don’t know you don’t know.

Type 1: Data you know (but probably don’t know well enough)

Descriptive Analytics use Key Performance Indicators (KPI).  Typical examples are leads, prospects, new customer growth, customer acquisition cost, etc. Online businesses are interested in website traffic patterns, time on site, customer retention, lifetime customer income, etc.
One reason investments in big data fail to pay off, is that most companies don’t do a good job with the information they already have.

Representative applications

Regardless of your business model, you should be interested in what the market and your customers are saying online, i.e. social business.

Social Business Tracking

Aggregation, Curation and Presentation

Data is just numbers but information is actionable. Several powerful applications allow you to capture and format data from various sources into actionable charts and presentations.

Maintenance and Real time analytics

Trend analysis help you fight fires but real-time monitors are smoke detectors – and there is wide range depending on your business objectives.

Type 2 – Data you don’t know that you don’t know

Predictive Analytics is exposing trends you don’t track, correlations you didn’t know existed, Connections you don’t know you have, and risks and opportunities that may be obscured.

Applications include:

  • Lead Scoring Uncovering additional leads from collateral data
  • Product Planning Identifying new markets for existing products and new products for existing markets
  • Networking Identifying new relationships from collateral data;
  • Dynamic Customer Profiling Real time behavior for online businesses.
  • Examples:

    Future Topics

    This series will continue on PageMill Marketing, Results Exponetialized.

    • Data Science – Big Data Case Studies
    • Marketing Automation – Integrated Platforms for Inbound Marketing, e.g. eMail, Lead Nurturing, etc., with interfaces to CRM
    • Big Data Case Studies
    • The leap from Data Science to Market Intelligence

    One Hundred Thousand Trains of Coal – updated

    The latest SEIA report for Q2 2014 shows faster than Moore’s Law growth in PV installations. Installation in 2014 will be 8X the installations just four years ago and doubling again by 2016. Local and state incentives combined with a continued drop in panels and installation infrastructure led the recent growth. Future growth will benefit from State mandates on utilities to increase renewable energy generation.

    The SEIA Year-End results for 2010, released earlier this month show a 878MW of solar photovoltaic installations in the US. Each MW of installed capacity will generate about 1.6GWHr of electricity per year which is the equivalent of burning 1600 tons of coal in a generator. Coal is shipped around the country on railroad trains. A typical train will carry ~11,000 tons of coal. So the installed PV capacity in 2010 will save (878 x 1600/11,000) or 127 trains of coal this year and every year for the next thirty or so years. Solar PV is forecast to growth at a 50% rate for the next several years, reaching ~10GW by 2020. At that time the cumulative base will be saving ~3800 trains of coal per year.

    If those numbers sound impressive, consider that the US consumes ~90,000 railroad trains of coal each year for electricity. How much solar or renewable capacity is needed to replace ALL of that coal. Let’s start with round numbers.

    • US consumption of coal ~1B tons ~600GW of solar PV
    • The average home PV system is 4.7KW. 600GW would require 127M residential PV systems – more than the current base of US homes!
    • At 15W/sq-ft, 600GW would require 40B sq-ft or ~10X the total commercial rooftop area in the US.
    • Solar currently costs ~$4/W installed. The DOE’s SunShot program is targeting $0.75/Watt by 2020. The cost of 600GW in 2020 would be $450B.
    • There are cells technologies with efficiencies above 30%. There are concentrator systems that increase power density more than 10X.
    • At 150W/sq-ft and $0.50/W, we could replace 100,000 trains of coal with 4B sq-ft of solar panels at a cost of $300B. Or about what we spend on foreign oil each year.

    Jobs in the 21st Century

    Either the robots work for you or vice versa

    1.7B people in the world have access to Smartphones. A device that can access all of the information in the world and show funny videos of cats. The Smartphone is not only changing how we learn but enabling new business efficiencies – farmers in 3rd world countries access weather and spot prices. But this is only the beginning of the IT revolution. When the next generation grows up, the smartphones we know today will be like pocket watches. The technology behind the smartphone – the application processors, the software ecosystem, the cloud infrastructure, data science, artificial intelligence, etc. – will change the way people live and work in unpredictable ways. Whether you are developing this new world or selling your personal skills, you will need to embrace the change to stay employed.

    But that doesn’t mean you need to code. Michael Polanyi (Polani’s Pardox) observed, “We can know more than we can tell…” In the field of artificial intelligence this is known as Moravec’s Paradox: “It is comparatively easy to make computers exhibit adult level performance on intelligence tests and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility.”

    The lower right corner – Non-repetitive skills – includes craftsman, artisans, oil field workers, healthcare workers, educators, rainmakers (people who sell complex solutions to people). This will likely be the dominant component of the work force. Of course the earnings potential will vary greatly.

    There is abundant evidence that repetitive skills are increasingly displaced by automation. The Internet disintermediation of interpersonal skills started a decade ago with travel agents replaced by Expedia, Kayak, etc. The trend is accelerating into every corner of the job market where goods and services once sold person to person, are now simple online clicks. Even taxi drivers and the hotel concierge are not immune to the power of ubiquitous Smartphones and perfect information.

    The upper right corner – thinking, non-repetitive – defines jobs for people who exploit technology to identify opportunities and solve problems. The McKinsey article below coins a new definition of CEO, as the Chief Experimentation Officer. Thinkers are experimenters. Smart Technologies are the “beakers and bunsen burners” of their cognitive laboratory.

    References and Recommended Reading

    Manager and the Machine, McKinsey Quarterly, September 2014

    Polanyi’s Paradox and the Shape of Employment Growth, David Autor MIT

    NAND Flash Ecosystem

    NAND Flash is the dominant non-volatile storage technology in portable electronics (Smartphones, Tablets). Solid state drives (SSD) are rapidly displacing hard disk drives (HDD) in laptops and moving into enterprise storage in a variety of configurations – flash cache, hybrid and full flash storage arrays.

    I created a an introductory slidedeck of the ecosystem circa mid 2014. You can view/download it from slideshare.

    Why the Aereo Supreme Court Decision is ultimately good for consumers

    You can read the full transcript of the SCOTUS decision here.

    Here is my simple interpretation.

    Over the air TV – which is strictly local – is free (unless you count your time spent watching Advertising). The local broadcasters make money by selling “local” ads. If a service can redirect the ATSC feed to a TCP/IP stream, then the value of local advertising is lost. In other words the content has been used for profitable purposes that do not benefit the content copyright owner or content license holder.

    So what about local TV over cable? Twenty years ago I had cable but I couldn’t watch local TV without an antenna. Then one day, the CableCo said “you can add local TV for $5/month because we cut a deal with the local broadcasters”. That became so popular it was soon part of the baseline package. All their subscribers got local TV and the CableCo paid the local broadcasters. Aereo on the other hand, offers a low monthly fee to capture and broadcast local TV without paying anybody. The CableCo’s and the broadcasters didn’t like that, hence the lawsuit that went all the way to the SCOTUS.

    People want to watch television content on their own terms, times and devices. There are solutions and there will be more choices, not fewer, as a result of this decision.

    0. If you subscribe to cable this is a non-problem – Xifinity on Demand from Comcast – but that doesn’t count in this context.

    1. The Major Networks – ABC, NBC, CBS, and FOX have free apps for iOS and Android smartphones and tablets and/or free and paid access through Hulu/Hulu Plus. Free means pre-roll and mid-roll ads but it only takes ~47 minutes to view a “one hour” show. Prime time is pre-recorded so why be a slave to a 10PM slot. Unfortunately many shows are delayed by hours or weeks.

    2. Apps for AppleTV / GoogleTV / Roku / SmartTV

    These can be ad-free under various subscription models. Google’s new Android L platform will significantly increase smart TV applications, surpassing proprietary approaches like Flingo. Apple is almost certain to provide similar incentives to both national and local broadcasters to create content apps for these devices.

    3. Personal DVR and home-casting.
    Content that arrives in your home legally can be archived and rebroadcast for you and your family’s personal entertainment. This is “fair use”. Slingbox is a perfect example of a legitimate solution. There are others, e.g. MythTV and XMBC are bring-your-own-hardware solutions. Boxee was a solution but they ran out of money after too many delays and pivots.

    Aereo’s defeat in the Supreme Court will increase the solutions along these lines. The bill of materials for a personal DVR that connects to your ATSC antenna and your home router is less than $100. I would not be surprised if the backers of Aereo have hedged their bets in this direction. “But I don’t have an ATSC antenna (or I live in an area with no reception. Aereo was my only choice.” Not true. In the Bay Area, Comcast, DirecTV and ATT U-verse provide local TV. Some provide a local channel only service for about what Aereo charged. IThe difference is that these companies “paid” for the rights. Aereo did not. And It is likely that Google and/or Apple will also license access to affiliate or even national broadcasters to provide yet another alternative.

    You can’t put the content behind a firewall and sell access any more than you can walk into a movie theater with a video camera or sell tickets to watch the World Cup on your big screen TV. The latter violation falls into the category of ridiculous to enforce, especially if your charge “beer”.

    Read / Update my wiki article on Web to TV

    Bitcoin Demystified (did I just pay $50 for a latte?)


    Myth: Bitcoins are not as secure as US Dollars

    Credit cards and financial account passwords are compromised everyday – whether by outright theft, hidden scanners or cameras that surreptitiously steal numbers, or hackers – causing millions of dollars in loss.

    Counterfeit paper money is an ongoing problem. And if the bank robber asks for small bills at the teller window, he will likely get to spend them.

    The Bitcoin “serial number” is registered and encrypted. Bitcoins can only be created solving complex cryptography problems in a competitive domain. It used to cost in electricity to mint a Bitcoin than they were worth. I doubt true now. And the cost to mine is dropping with electronic hardware (ASIC and FPGA) mining solutions.

    Myth: Bitcoins markets are run by criminals and cater to criminals

    If you don’t have the computer skills to mine Bitcoins, you must buy them through an exchange. Years ago, MtGox was the only option. Registration requires levels of identify verification as well as verification of funding sources. Trading on Mt. Gox is imperfect and randomly closed. On the other hand US stock markets have relatively narrow trading hours while BitCoins are traded 24×7.

    Exchanges can be frozen or even shut down by government authorities. The use of Bitcoin markets to trade illegal goods or fund illegal activities is a valid concern. The FBI shut down TheSilkRoad and arrested the organizer. The news produced a brief 20% drop in BitCoin value, which became the buy opportunity of the year. The FBI isn’t going to dump Bitcoin accounts needed for a trial years in the future. The US government restricted Bitcoin usage on digital payment sites like Dwolla The Chinese government has made similar moves recently. On the other hand, a lot of people lost money when the US Government shut down online gambling sites, e.g. PokerStars and FullTiltPoker. Comments by government officials this week portend a positive environment for Bitcoin and other digital currencies. See Bloomberg news

    A number of BitCoin exchanges and marketplaces have been funded in the last few months. Most of these are backed by Tier 1 venture capital funds with deep pockets and a long-term horizon.

    • BitInstant (New York): Platform for instantaneous Bitcoin transfers.
    • Bitpay (Atlanta): Lets customers of businesses ranging from software-makers to auto dealerships make payments in Bitcoin, then transfers cash to those merchants.
    • Coinbase (San Francisco): Makes a “virtual wallet” that lets users buy Bitcoin and pay for goods and services with it.
    • CoinLab (Seattle): Backed by Silicon Valley investor Tim Draper. Recently tried teaming with leading Bitcoin exchange Mt. Gox; the partners, however, are now in court.
    • Lamassu (New Hampshire): Has developed an ATM to instantly turn dollar bills into Bitcoin.
    • OpenCoin (San Francisco): Cofounded by E-Loan’s Chris Larsen, it has created a payment system to transact various currencies, including dollars, Bitcoin and a new alternative, Ripple.
    • Tradehill (San Francisco): Runs a Bitcoin exchange that competes with Britain’s Bitstamp, Russia’s BTCE and Tokyo-based Mt. Gox, among others.

    If exchanges make you nervous, simply use them for conversions, then store your BitCoin in a personal secure, encrypted wallet. I use BitCoin-Qt.

    Fact: Digital Money is here to stay

    When is the last time you saw someone write a check in the grocery store, especially someone under 60? Today’s kids will shop with their smart gadget, not with a credit card and certainly not with folding money as my grandfather called it.

    Bitcoin has advantages for micropayments (the smallest unit of payment is 1 Satoshi = 0.00000001 BTC) and that amount can be transferred at no cost. Independent artists could sell content for fractions of a penny without losing the $0.30 + 3% that Paypal charges.

    Bitcoin is the first digital currency to gain a foothold. But it could be supplanted by something better. There is a gold analogy with respect to Bitcoin – the more that is mined, the harder it gets to mine more. Gold costs $300 per ounce to mine which sets a base value. Bitcoins mining essentially has half-life. About half of the maximum possible Bitcoins (21,000,000) has been mined. The remaining 11 million will be fully mined by 2017. This has several implications.

    The US M1 money supply is ~$2.5T. In simple terms replacing M1 dollars with M1 Bitcoin equivalent would equate to a value per Bitcoin of $100,000. Most goods would be priced in mBitCoins or uBitCoins which sounds awkward. (1 MicroBitCoin = 10 Satoshi’s if you’re keeping score).

    BitCoin software is open source. What if the US Treasury began minting DigitalCoins under a derived scheme, that combined US Government money management with the cyber protection advantages of BitCoin. Would Mt. Gox stand a chance?

    Did I just pay $50 for a latte?

    Success of Bitcoin hinges on the use of the digital currency for buying and selling legal goods and services. The price gyrations of the last few months make that a daunting challenge. Similar to the problems with runaway inflation, how much does an item really cost if the dollar basis is fluctuating by 30% to 90%. And that leads to the second problem – lack of merchants that accept BitCoin. At the end of the day, the rent has to be paid in dollars (or the local currency).

    What is needed is a more efficient marketplace. Farmers hedge against price drops in crops on the Futures Market. Traders hedge again stock drops (or pops) with options. If people can make money regardless of the price movement direction, the forces that accelerate bubbles are diminished.

    Full Disclosure

    I gave my nephew a BitCoin for his 14th birthday. He sold it to buy an XBox One. I hope it doesn’t turn out to be a $50,000 game box.

    SmartPhone, SmartTV, SmartEverything


    In only a few years, the Smartphone has become the personal computer of over one billion people. Another billion will be added in the next few years. The success is not merely due to the electronics in the phone, but the infrastructure of cloud services, applications and storage that allow personal computing devices to pervade and enhance our lives.

    The technology inside today’s smartphones is comprised of ARM-based and Intel CPUs, graphics and video engines (GPUs), Mobile Interface IP, Memory components, peripherals, and wireless access devices (e.g. Cellular, Wifi, Bluetooth). Mobile Interface IP is governed by Standards, and optimized for low power consumption with high performance.

    This same technology is now moving into a range of products beyond phones – powered by batteries or the grid – like wearable accessories, SmartTVs, SmartCars, SmartHomes, etc. These products will expand rapidly as the ecosystem evolves. The common denominator in these products is the same mobile computing Silicon IP that powers Smartphones. The AppleTV is just an iPhone without the cellular radio. The Samsung Chromebook has the same chipset as the Samsung Galaxy S3.

    Read the full blog post on Arasan.com

    Advertising Insights

    Advertising is Emotional

    “The most successful advertising campaigns are not those we love or hate, or those with messages that are new or interesting, but those that are “able to effortlessly slip under our radar and influence our behaviour without us ever really knowing that they have done so…”
    from Seducing the Subconscious by Dr. Robert Heath

    Search Advertising is Location Dependent

    Competition Spurs Performance in Mobile Storage

    Many companies were surprised by the unexpected eMMC developments last November, when Samsung announced availability of a 260MB/s eMMC device. The announcement roiled the mobile storage industry and led JEDEC to hastily finalize a revision to the eMMC specification. The result is eMMC 5.0 with a peak transfer rate of 400 MB/s (equivalent to 3.2 Gbps). This moves eMMC slightly ahead of UFS v1.1 with 2.9 Gbps transfer rate. To further blur the advantage of UFS, some advanced features originally intended for UFS are now been planned for future eMMC revisions.The new eMMC 5.0 specification will also be introduced in the 3rd quarter of this year; its maximum data transfer rate will be 400MB/s or 3.2 Gbps in 8-bit bus operation. This high data transfer rate fills the performance gap before UFS entire ecosystem is available. In addition, an eMMC version of command queuing is now on eMMC roadmap for a future specification revision. It is also possible that eMMC will continue to increase the maximum data transfer by another factor of two.

    Is UFS adoption in tablets and smart phones threatened by the recent advent of eMMC 5.0?

    UFS version 2.0 will be introduced in the 3rd quarter of this year; its maximum data transfer rate can be 11.6 Gbps in a two-lane configuration with 5.8 Gbps per lane. In addition, native command queuing, inherited from the high performance SCSI protocol has been added. This provides multi-thread or multi-tasking operations which can provide an order of magnitude performance advantage over eMMC. A special extension has been added for Uniform Memory Access (UMA). This feature will allow non-volatile memory devices to extend on-die buffer storage for L2P (logical to physical) mapping. This can improve performance and device lifetime.

    It seems like eMMC is positioned to fill the performance gap before UFS ecosystem (HW, SW, OS support, development tools, etc.) is ready. As a new technology, UFS will initially be adopted by high-end mobile devices. As the benefits of UFS are fully demonstrated in initial products, and the technology matures with increasing volume, UFS adoption will spread. UFS is supported by UFSA, a governing body of industry representatives chartered to define and certify compliance to the UFS standard.

    UFS relies on the MIPI M-PHY. The M-PHY V2.0 supports transfers of up to 5.8Gbps (Gear 3) and is being adopted by additional standards bodies, e.g PCISIG for mobile PCIe, and USB for SSIC. This will further lower barriers to UFS adoption in high bandwidth, power critical applications.

    Written by