Data Overload in the Age of Excess

Key Takeaways

  • The amount of data we generate, consume, and gather has never been higher.
  • Government funded and publicly available data has spawned entire industries.
  • Accessing publicly available data is often a difficult and frustrating process, but it doesn't have to be.
  • Voyager Search is developing a new tool to make publicly available data easy to find and use.

Data is everywhere. Companies bemoan the overwhelming amount of information they have; our government is constantly gathering data and producing research from it; and the media are generating more content than ever. The subject of data is omnipresent. But how did we get here, and how do we convert this information to actionable knowledge?

The United States government makes data publicly available because it’s funded by taxpayers’ dollars. By doing so, the government remains transparent, and, opportunities open up for increased efficiency, innovation, research, and development.

Free information has spawned entire industries. Take weather and climate data, for example.  Most people are familiar with the National Weather Service (NWS). The NWS enables a thriving weather industry worth $7 billion dollars, and it’s estimated that this industry unlocks another $35 billion dollars in economic value. Why? One reason is that people don’t often go to the NWS for information. Instead, companies have taken the NWS data and built consumer-friendly applications such as Weather Underground, or business services like AccuWeather. There’s also an entire media industry that is the result of this weather data — think The Weather Channel, your local nightly weather report, and much, much more.

The same can also be said for Census data. The Census that our government conducts every decade is the basis of all the demographic data companies use today for target marketing, yet no one goes directly to the folks in charge of managing the Census. Marketers do, however, rely on companies like Dun & Bradstreet and to provide them with information on who lives where, how many people live in the household, and how old they are, in order to determine the best place to  spend their marketing dollars. Census data is also used for planning transportation corridors and infrastructure development.

While not all governments around the world have data mandates, they’re starting to see the benefits of allowing others to access their data. At minimum, it’s a way to promote their own work and tools, while creating an open, collaborative environment. Massive industries that we as consumers use every day are all tied to the accessibility of data.

Why Is Data So Hard to Find?

With so much data available, people and companies often struggle to find what they need. The amount of data is simply overwhelming. One contributing factor is that the tools that are used to publish data are not always searchable by search engines. GIS professionals know this struggle all too well. Finding the right topographic layer for your map, the appropriate LiDAR imagery for a certain time frame, or the correct DEM file for a specific location is often a time consuming and frustrating process. Search engines aren’t applying geotags when indexing, so GIS data become difficult to find. One way to solve this problem is to embrace the fact that people publish data in all kinds of different ways - from FTP, mapping web services, and varying data sets, to web protocols, downloadable mine types, and portal technologies. And that’s just naming a few.

At Voyager Search, we know there’s a better way to search. We’ve been working on a solution, a new approach to searching publicly available data. We’ll be sharing our findings with you in the coming months so be sure to check back for continued updates. In my humble opinion, it’s the most exciting time to be in the business of search.


Brian Goldin

Founder and CEO of Voyager Search

Web Design and Web Development by Buildable