• Home
  • Diver BI Group
    • Diver | BI
    • Customer Commendations
    • Videos
    • Case Studies
    • Documents and Brochures
    • Diver BI Group Privacy Policy
  • Articles
    • News
    • Accolades
    • Blog
    • Ask Anthony
  • The Team
    • Key Experts
    • Members
      • Dynamic Business Informatics
      • Envisage
      • Chartwell IT
      • Cloud-Works Solutions Ltd
      • Data Management Business Insight
      • Manitex
      • Equiniti ICS
      • Pharma IT
      • Pinnula
      • Formell
      • IBI
    • Partners
      • Dimensional Insight US
      • Dimensional Insight EU
      • Dimensional Insight Asia
      • DI Panama
      • ID Technology (Brazil)
      • Infotool (Nordic Countries)
      • BIMAXIS – DI France
      • Vision IT
      • KDE Group (Israel)
      • Zight -Partner for Transport
      • 24/7 Uptime
  • Applications
    • Property
      • Property Prices Portal
    • Supply Chain
      • Retail
      • Distributor
      • Wholesale
      • Manufacturing
      • Produce
      • Beverage Alcohol Suppliers
      • Beverage Alcohol Distributors
      • Autoparts
      • Transport
      • Public Transport
      • DI Program Advisor
    • Healthcare
      • Healthcare
      • DI Surgery Advisor
      • DI Physician Performance Advisor
      • Pharma BI Applications of Diver
    • Finance
      • Finance
      • DI GL Advisor
    • Mobile & SaaS
      • Diver BI SaaS
      • SaaS for Sales Gap Analysis
      • BI designed for Tablet
      • DiveTab – Mobile BI
    • Diversity
      • Travel & Tourism
      • Colleges & Universities
      • Waste Management
      • Parking
      • Telecom Brokerage
      • Economic Trends
      • Maps
      • Smart Meters
      • Carbon Emissions Management
  • Workbench
  • Contact
    • Free Consultation with One of Our Experts
    • Add a New Revenue Stream as a Partner

DiverBIGroup

Business Intelligence Experts

Diver BI Group Blog Practical Analysis: The Next Chapter

Practical Analysis: The Next Chapter

3rd June 2020

by George Dealy | May 21, 2020 

The “Practical Analysis” blog series is dedicated to answering the questions: “So what is analysis anyway?” and “How can I apply it in my world?” The first installments focused on the fundamentals from which modern computer-aided analysis and decision support have evolved. They featured topics like visualization, exploratory data analysis, and statistical process control.

In the next posts, we’ll explore the emergence of what we now know as “data science” from those beginnings. Let’s examine.

Tukey’s impact on data science

My research into analysis kept invariably pointing me in one direction and I think that is an appropriate place to start this chapter. In 1962 John Tukey, one of the most prolific mathematicians and statisticians of the 20th century, wrote a provocative article in The Annals of Mathematical Statistics titled “The Future of Data Analysis.”

Tukey’s intent was to take the mathematical statistics community, of which he was a member, to task for their nearly exclusive focus on a highly theoretical approach that offered limited practical value. Much of the academic research in statistics to that point had been consumed with creating inferential models from relatively small data sets and understanding the limitations of their reliability.

But meanwhile, early computing technology was quickly evolving in ways that made increasingly larger datasets both possible and practical. For example, the introduction of punch card machines helped to automate the U.S. decennial census leading to one of the first large, well curated, and reliable datasets.

Part of Tukey’s argument was that data itself held the promise of illuminating many additional theoretical possibilities. Furthermore, this approach could spawn practical applications significantly beyond those that had arisen from the obsession with small data.

Tukey’s initial training as a chemist equipped him with the dual perspectives of rigorous theory and an evidence-based approach to applying theory to practice. In The Future of Data Analysis, he drew on the physical sciences for examples of how data analysis could evolve as a science of its own. Understanding the properties and patterns of data in general could help to facilitate discoveries across disciplines. He even went as far as to claim that data as a science could be among the most complex and challenging areas of research.

It’s interesting that though Tukey has been credited with coining the computer age terms “bit” and “software,” it would take more than a decade for “data science” to enter the technical vocabulary even though the field quickly took shape more or less how he had predicted.

Innovations lead to modern data science

If you’d like to understand the emergence of data science from its roots in The Future of Data Analysis, I’d suggest reading the paper “50 Years of Data Science” by David Donaho. Donaho, a professor of statistics at Stanford and one of John Tukey’s many proteges, gave the lecture on which the paper is based during a conference celebrating the centennial of Tukey’s birth and his contributions to math, science, and society.

Donaho’s paper chronicles the innovations in both the academic and industrial realms that have led to today’s data science. One dynamic that likely accelerated these developments was that Tukey had a foot in both camps: he was the founding chairman of the statistics department at Princeton University as well as a senior researcher at Bell Labs, one of the foremost research institutions in the world at the time.

On the academic side, Tukey and his collaborators pioneered the field of exploratory data analysis while also refining existing statistical methods such as regression and analysis of variance. Simultaneously, his associates at Bell Labs were applying those theories in domains such as telecommunications networks and semi-conductors while also developing the software technology that would serve as the foundation of modern machine learning. The S statistical language, a predecessor to open source R language, was developed at Bell Labs by Tukey associate John Chambers. It has played an important role in the progression of software tools that have paved the way for applying advanced statistics to ever larger datasets.

A recurring theme in the work that Tukey undertook during his career is the impact of data and evidence-based thinking on issues that affect everyday life. These ranged from his role in investigating the methods used by Alfred Kinsey in his research into human sexuality to the effects of pesticides on the environment to the accuracy of Census Bureau population estimates and the reliability of predicting election results based on partial and rapidly changing information. These are all inspiring and instructive examples of how data-driven decisions can and do shape our world.

Inferential and predictive approaches to statistical modeling

Among the ongoing debates that 50 Years of Data Science highlights is the tug of war between the inferential and predictive approaches to statistical modeling. The goal of the inferential approach is to develop generalized, or generative, models that explain the transformation between a set of inputs and the results that reflect a set of outcomes.

Prediction, on the other hand, harnesses patterns in the actual data to project the likelihood of future results in an evolving series of events. While both techniques can be effectively applied to a wide range of problems, the predictive field has played a particularly influential role in the development of modern data science, providing many of the mechanisms that underlie today’s highly scalable machine learning and artificial intelligence systems.

And it is prediction that is really at the heart of what John Tukey envisioned in The Future of Data Analysis: using data to understand the phenomena that they represent and better realize the opportunities they embody.

50 Years of Data Science concludes with an examination of the Common Task Framework, a community-based, open source approach to developing progressively more effective predictive models through the use of shared datasets, well defined goals, and objective evaluation of results. The Common Task Framework is yet another step toward realizing Tukey’s vision of treating data and analysis as legitimate and compelling science.

In subsequent posts I’ll explore the algorithms, including both inferential and predictive techniques, that trace the evolution of data science and its role in fostering increasingly pervasive and effective data driven decision-making.

George Dealy is a member of Dimensional Insight’s leadership team.

Tags: Data Analysis, Data analytics, Data Science, George Dealy, Inferential analytics, Predictive analytics

Articles

  • News
  • Accolades
  • Blog
  • Ask Anthony
  • Diver | BI
  • Customer Commendations
  • Videos
  • Case Studies
  • Documents and Brochures
  • Diver BI Group Privacy Policy

Dynamic Business Informatics Ltd

Distributors of The Diver Solution & Diver Platform, providing business consultancy services, products and applications derived from the Diver Solution & Diver Platform.
14 Mellifont Avenue
Dun Laoghaire
Co. Dublin

+353 1 2302022

10 Kenyon Street
Nenagh
Co Tipperary

+353 67 43309

Contact Us

We would love to hear from you! Please submit your query and we will get in touch with you shortly.

Email addresses are never sold, rented, or shared. See our Privacy Policy.

  • This field is for validation purposes and should be left unchanged.

Accolades

  • Dimensional Insight Named an Overall Leader by Dresner Advisory Services in 2022 Industry Excellence Awards
  • Dresner Wisdom of Crowds 2022 – Best in Class Results
  • Klas 2022 – A Grades for Dimensional Insight
  • Dimensional Insight Earns Top Marks in 2021 Wisdom of Crowds Business Intelligence Market Study
  • Best in Klas 2021

Copyright © 2023 Diver BI Group · Privacy Policy · website by cubecolour

We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
Cookie settingsACCEPT
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.


What Are Cookies?


As is common practice with almost all professional websites this site uses cookies, which are tiny files that are downloaded to your computer, to improve your experience. This page describes what information they gather, how we use it and why we sometimes need to store these cookies. We will also share how you can prevent these cookies from being stored however this may downgrade or 'break' certain elements of the sites functionality.


For more general information on cookies, please read "What Are Cookies".


How We Use Cookies


We use cookies for a variety of reasons detailed below. Unfortunately in most cases there are no industry standard options for disabling cookies without completely disabling the functionality and features they add to this site. It is recommended that you leave on all cookies if you are not sure whether you need them or not in case they are used to provide a service that you use.


Disabling Cookies


You can prevent the setting of cookies by adjusting the settings on your browser (see your browser Help for how to do this). Be aware that disabling cookies will affect the functionality of this and many other websites that you visit. Disabling cookies will usually result in also disabling certain functionality and features of the this site. Therefore it is recommended that you do not disable cookies.


The Cookies We Set




  • Account related cookies


    If you create an account with us then we will use cookies for the management of the signup process and general administration. These cookies will usually be deleted when you log out however in some cases they may remain afterwards to remember your site preferences when logged out.




  • Login related cookies


    We use cookies when you are logged in so that we can remember this fact. This prevents you from having to log in every single time you visit a new page. These cookies are typically removed or cleared when you log out to ensure that you can only access restricted features and areas when logged in.




  • Forms related cookies


    When you submit data to through a form such as those found on contact pages or comment forms cookies may be set to remember your user details for future correspondence.




  • Site preferences cookies


    In order to provide you with a great experience on this site we provide the functionality to set your preferences for how this site runs when you use it. In order to remember your preferences we need to set cookies so that this information can be called whenever you interact with a page is affected by your preferences.




Third Party Cookies


In some special cases we also use cookies provided by trusted third parties. The following section details which third party cookies you might encounter through this site.




  • This site uses Google Analytics which is one of the most widespread and trusted analytics solution on the web for helping us to understand how you use the site and ways that we can improve your experience. These cookies may track things such as how long you spend on the site and the pages that you visit so we can continue to produce engaging content.


    For more information on Google Analytics cookies, see the official Google Analytics page.




More Information


Hopefully that has clarified things for you and as was previously mentioned if there is something that you aren't sure whether you need or not it's usually safer to leave cookies enabled in case it does interact with one of the features you use on our site.


However if you are still looking for more information then you can contact us through our contact page

Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT