This blog post explores how data mining technology is changing our daily routines and what role it will play in future society.
Everyone has probably missed the last bus or subway at least once and ended up taking a late-night taxi home. In such situations, the taxi fare is burdensome, but with no alternative, it becomes an unavoidable choice. However, to address this inconvenience, long ago, the Seoul Metropolitan Government launched a late-night bus service to enable public transportation use during late-night hours. This decision is evaluated as a successful case of data-driven transportation policy, going beyond simply providing convenience. The late-night bus routes were carefully determined based on extensive data regarding traffic volume at major subway and bus stations, not mere guesswork. Born from this scientific approach, the late-night buses gained significant public approval and also contributed to alleviating traffic congestion.
Furthermore, when searching for books on online shopping sites, you often encounter sections like ‘Customers who viewed this item also viewed’ or ‘Personalized Book Recommendations’. We often discover and purchase unexpected books through these recommendation systems. In this way, data mining technology has become deeply embedded in our lives and is gradually expanding its influence into more fields. Currently, data mining technology is applied across diverse sectors including public transportation, online shopping malls, internet search engines, social networking services, and the financial industry. Data mining refers to the process of processing and analyzing data based on statistical theory and computational power to derive meaningful conclusions. These conclusions bring tangible changes and benefits, such as analyzing individual consumption patterns or supporting corporate decision-making. So, what changes will data mining bring to our lives?
Data is the product of processing the world and transforming it into a form that is easier for us to understand. Data can exist in any form—text data composed of written language, photo data, as well as audio, video, and more. For instance, the vast amounts of text data generated on social media can be utilized to analyze public sentiment on specific topics. The term “data mining” emerged from combining “mining,” meaning extraction, with “data.” The goal of data mining is to find relationships and patterns using data. As a field within industrial engineering, data mining reflects the core of industrial engineering—analyzing specific subjects statistically and computationally to understand their characteristics.
Text data can be obtained from posts users make on social networks, while numerical data can be derived from stock price changes in financial markets. How can data gathered from such diverse sources be processed? This is achieved through the methodology of statistics and the tool of programming. Two main tasks are performed: removing unnecessary data and conducting statistical analysis on the remaining data. This process is akin to meticulously shaping a rough stone into a gem. Results derived through data mining can be utilized in various forms, such as predictive analytics, decision support, and marketing strategy development.
How can this obtained data be utilized? Data mining influences not only our daily lives but also societal and national-level changes. For instance, Google, a leading search engine, attempts to predict broader societal shifts by analyzing the frequency of specific search terms. A sudden surge in searches related to the common cold—such as “cold,” “mask,” or “cold medicine”—can indicate an impending widespread cold epidemic. These data analysis results can play a crucial role in establishing public health policies and provide a foundation for governments and institutions to respond swiftly.
Since data can exist in any form, there are no limits to the areas where data mining can be applied. ‘Natural language processing technology,’ which has been continuously active over the past decade, holds significant potential value. Scholars are developing universal translators using massive amounts of data. Even as we speak, researchers continue their work to create a world where people can communicate regardless of language. If language barriers disappear, the world will transform in ways unimaginable today. Furthermore, AI robots capable of natural conversation using human language will be developed, serving as companions for the elderly as society ages. This is not mere imagination but technology already partially realized in reality. The healthcare industry, which analyzes real-time bodily signals to provide personalized medical services, will expand significantly. Data mining technology will also flourish in numerous fields, such as criminal investigations that track offenders using various CCTV data networks.
Thus, we have explored the question of what data is, the process of data mining, its application areas, and its future development potential. As French philosopher Marcel Proust said, ‘True discovery is not about finding new lands, but seeing with new eyes.’ Similarly, data mining technology will become the eyes that allow us to see the world anew. In future society, data mining will be not just a simple tool, but a core driving force creating a new paradigm.
