Writers Workshop: Big Data For The Big Guy

Image by Pete Linforth from Pixabay

A couple of weeks ago, I talked about being the top graduate in Production and Operations Management in my class and earning the Production Management key for my trouble. One of the things that attracted me to the major was that it involved a lot of statistics and number-crunching, In fact, the classes in the major were all given the designation of OMQM, for Operations Management and Quantitative Methods. I couldn’t have cared less about the Operations Management side, it was the Quantitative Methods that I was really interested in. These included the courses in Statistics and Data Processing (the old name for Information Technology, or IT). You can take the boy out of Math, but you can’t take Math out of the boy.

Problem was, there was no real career path for statisticians at the time. I looked into actuarial science, but that required a lot more math than I had, and I really didn’t want to go there. I considered becoming a baseball statistician, particularly with the advent of sabermetrics, which created a need for statistics that was above and beyond batting average and ERA, but that would have required moving to New York, where the offices of the major leagues and most of the companies that computed those numbers were located. There was always government work, and… just no. So I moved into Data Processsing and had a good career there, as a programmer, technical support specialist, database administrator, and trainer.

Lately I’ve been seeing a lot of ads on Instagram (which has a lot of ads, anyway) for a relatively new field called Data Science, part of the ever-growing field of Big Data. The Internet has made it possible for businesses and government entities to collect massive amounts of data, and Data Science looks for ways to attach some meaning to the numbers. Granted, it’s a scary prospect, knowing that there is so much data floating around out there, some of which has to do with each of us, evaluating areas of our lives that we didn’t even know existed, and many of the ethical considerations have yet to be evaluated. Still, the more I see about it, the more interesting it becomes. It’s a very new field which evidently didn’t even exist until a few years ago; still, I have a good background in statistics and statistical methods, and it would be great to get in when the field was relatively new.

So, that’s what I’d do: if I had to choose a new career for myself, I’d be looking at Data Science.

18 thoughts on “Writers Workshop: Big Data For The Big Guy

  1. My comment is data-based but a step outside of what you wrote about. Actually, I’d love your expert opinion on something that was shared with me over the week-end, by a Holistic Practitioner I work with; it involves the entire “ancestry” data-gathering, genetic research services.

    This practitioner, as an experiment, submitted swabs that were dipped in plain water. The results came back with a report of various genetic backgrounds, with a high percentage of…Lithuanian. That being stated, and back to your post, has the progression of data collection given way to, perhaps, incorrect, even fraudulent results by some genetic research services? Again, just your well-educated opinion, John.

    Like

    1. Could be that Vitus or Ruta in the lab contaminated the sample…

      But seriously…

      Using DNA to determine ethnic group is still relatively new. I had mine checked at Ancestry several years ago, and it came back that mine showed a trace of Polynesian. About a year later, they contacted me and said they had checked my DNA and adjusted the figures, and Polynesian was gone. 23andMe, who does the DNA testing for “Finding Your Roots With Henry Louis Gates” on PBS, attached a probability to my DNA, saying that they were 90% sure I was Irish, but if you take a lower probability (say 75%), there’s more room for error, that the DNA might indicate I’m Irish, or that I’m mostly Irish with a fairly large percentage of indicators that some of my ancestors were from other parts of the Britsh Isles, or that my ancestors were mostly from Ireland and the British Isles but there were also indicators that suggest some of my ancestors were from northern Europe, and there might have been a few from other parts of Europe. The DNA testing done by Ancestry and 23andMe is much better for locating relatives: Ancestry told me that I was very closely related to my mother’s sister (duh), and 23andMe put me in touch with my grandmother’s brother’s children.

      Sorry for the longish aside…

      Anyway, while water itself doesn’t have DNA, microbes, bacteria and other living organisms live in the water, as does mitochondrial DNA (dead skin cells etc.) from people who used the water before you. (Water treatment, the process of turning dirty water into water that’s suitable for drinking, cooking and bathing, and ensuring there’s no sudden outbreak of cholera or other water-borne diseases, does a good job, but it’s not perfect.) It’s possible that the lab picked up DNA from one of those sources and analyzed it.

      Is it quackery? I think it’s limited in what it can and can’t do, and that, as more samples are received and analyzed, they’ll be able to learn more from it. Even then, it’s a best guess.

      Like

  2. So, the career choice you would make IF? Do you think you might? Shaking things up can be so invigorating. I will be interested to see if this idea goes any where over time.

    Like

  3. Baseball now is all about data research. The front offices are full of data people who pull it on every high school, college and minor league player. That I would love to do.

    Like

      1. There you go John! That would be a job I would love. Football is starting to follow baseball and they are doing it much more than a few years ago.

        How many times X struck out on a 2-2 count etc…

        Like

  4. Do you like to cook? I have been intrigued by the number of technically-oriented people I have known
    who love, or loved, to cook. My theory is they all enjoy manipulating numbers or things and arriving at a finished solution or product which is what a good cook does with ingredients. Since you mentioned your interest in Data Science and other technical pursuits, I got to wondering – Do you like to cook?

    Like

    1. The thing was that, in statistics classes, you did all the number-crunching yourself. Now it’s all done by computers, so the trick is to know how the software works and how to use it to get what you need. Of course, a good understanding of the different tests you can peform always helps…

      Liked by 1 person

You can use Markdown in your comments. Thanks for your comment!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s