Data, technology critical legs of the large-scale event planning stool

When preparing for a large-scale event like a festival or the State of the Union, the past is prologue to the future and the future may be planned for based on what you know today and tomorrow.

This is why agencies, at the federal level as well as the state and local levels, are turning more toward open source information to drive decision making and planning.

A new research paper by the Center for Technology in Government at the University at Albany (CTG UAlbany) found four important qualities that public sector organizations need to drive better decisions.

• Data: data quality and coverage, compatibility and interoperability, and external data
• Technology: information systems and software and analytical techniques
• Organizational: cooperation and culture
• Institutional: privacy and confidentiality, and public procurement

This research highlights steps agencies need to consider as they drive deeper into these approaches.

Jay Humphlett, the executive vice president for public sector at Dataminr, said the data and technology  pieces are two areas the industry has been working on for many years to help federal, state and local governments make better, faster and more accurate decisions.

“Because of the sheer volume of open source information, you need artificial intelligence to help cull through all that and generate the critical alerts and get them delivered. We always pride ourselves on providing accurate alerts to the right individual at the right time,” Humphlett said on the discussion Using real time information to prepare for large scale events. “We operate exclusively in the publicly available information continuum. All we provide is open source information. We don’t do anything in the classified realm, whether it’s a dataset or our output. The open source information landscape is very broad, and it’s changing every day. There are new datasets that are being created. So, part of our challenge is how do we stay on top of that, identify the datasets that can be performative for our solution, and then integrate them in.”

He said Dataminr pulls from over 300,000 datasets which include global and hyper-local social media sites, blogs, the deep and dark web and other publicly available sensors.

Humphlett said the company plans to expand that number of datasets to 500,000 in the coming year.

Simply understanding that there’s a breadth of public data is just one component of preparing for planned and unplanned events, but the technology has to be fine-tuned and be able to narrow down what’s relevant in order to drive that effective and fast decision making.

“It is artificial intelligence, machine learning, deep learning neural networks and on and on. We have very robust AI solutions. We use 1,000s of different bespoke algorithms that we’re able to employ to go through all of the data and then to generate those alerts in real time,” Humphlett said. “We are bringing in billions of data units every day and evaluating them in real time. We don’t want to overload our users. We have to make sure that we get the signal-to-noise ratio correct and tuned to their particular mission.”

He added tuning the data and AI algorithms to the user’s needs is something Dataminr spends a lot of time on. He shared that using various technologies to configure alerts and distill public information is part of how they ensure services are flexible enough to meet agency needs.

Another big challenge is the data taxonomy and how that translates into alerts.

Humphlett said Dataminr works with federal, state and local agencies, as well as other organizations around the world to make sure they achieve “information symmetry,” so users more easily understand what is happening in their county or region or state.

“We did a return on investment study for our private sector clients about essentially the Pulse product that we have for our corporate risks. I believe it’s over 400% ROI within six months, and it’s just the return on investment,” he said. “We are providing this valuable real-time alert stream and allowing these corporations to make these real-time decisions much faster. Whether it’s for executive protection, asset protection, brand awareness, whatever it may be, they’re able to derive that benefit very quickly.”