By Joshua Steier
Technology & Innovation Writer
“Hiding within those mounds of data is knowledge that could change the life of a patient, or change the world” (Atul Butte, Stanford School of Medicine). Big data is transforming daily life, and lies at the heart of many companies such as Google and IBM.
Big data is essentially, using predictive analytics on large amounts of data. It consists of the following characteristics: volume, variety, velocity, variability and veracity.
A key difference between big data and other data is that big data needs to be monitored in real time, and is therefore updated much more frequently than regular data. The implications are also much different, since the sheer complexity of big data is much more massive than regular data. The usage of big data is so ubiquitous that its use cannot be overestimated. Big data is positively impacting research in astronomy, and medicine.
Research in astronomy has significantly improved since being involved with big data. Researchers have constant flows of data in the form of pictures that they need to sift through. This problem has important implications in the field of computer vision. Computer vision focuses on images and utilization of those images in computer software.
It’s mainly utilized for facial recognition techniques with a variety of software available such as, Python, MATLAB and Octave.
The algorithm developed to deal with big data classifies images so it can sift the large amount of pictures.
Classification is an essential problem in a topic known as machine learning. The objective is to either classify or to interpolate on some data. Classification of these pictures seems simple, but it is a complicated process. An algorithm is created from using a training and a test set of data. The training data is inputted into the machine to create the algorithm and the test set is tested on to determine the accuracy of the algorithm. The algorithm created for astronomy research can be also used for cancer research, the body is like the galaxy, and the various anomalies are like the stars.
Therefore, the big data involved with the research in astronomy can also be used to detect cancer in patients. This is not the first time big data has crossed with medicine. For instance, IBM Watson is currently being used to offer cancer treatments for patients. IBM Watson originally was celebrated for winning Jeopardy. Watson actually has access to a large variety of data, and therefore utilizes big data techniques for answering questions.
According to Dr. Thomas J. Marlowe of the Mathematics and Computer Science department at Seton Hall University: “Cancer research stresses all of the major facets of Big Data: data management, analytical techniques from many areas, and visualization.”
Dr. Marlowe gave a lecture titled: “An overview of Big Data” previously, and is offering a course for Big Data in the fall.
The algorithm used to identity patterns in the cosmos can be used to identify patterns within cells, which can lead to a new direction in cancer research. In other words, the big data algorithms utilized for astronomical research has led to a breakthrough in utilizing big data also for cancer research.
Big data would allow for real time data processing and analyzing, thus paving the way forward for a futuristic society. Even though big data can improve our lives, there are a few problems that need to be resolved.
For instance, we need more specialists equipped with knowledge of big data to implement algorithms and understand the analysis. Furthermore, more research into big data as a field needs to be done so that it can be implemented properly.
According to Dr. Thomas J. Marlowe: “More research awaits in all three areas—data management, data analysis, and data visualization, as well as in improving the underlying computing platforms and their performance.”
Lastly, according to Dr. Manfred Minimair, professor of Computer Science, and pioneer of the DAVA(Data Analysis and Visualization) Certificate offered at Seton Hall University: “Being able to analyze Big Data is of great importance to advance science and taking advantage of business opportunities alike.”
In other words, big data is an essential part to the future and will impact many research areas, such as astronomy and medicine.
A version of this article appeared in the Tuesday, March 21st, 2017 print edition.
Contact Joshua at