New MeriTalk Study Finds State and Local Agencies Not Prepared to Leverage Big Data

By Peter Bernstein May 01, 2013

One of the ways I stay on top of industry trends in the U.S. is by monitoring what is going on in the government sector. The reason is that not only are governments really big spenders on IT in general, but due to their very nature they spend a lot on the creation, storage, management and analysis of an extraordinary amount of structured as well as unstructured data. State and local government IT professionals, like their colleagues in the enterprise world, are currently staring at a data explosion. A question regarding these big challenges is whether they have a strategy for using all of the capabilities of “Big Data” to improve decision making and meet mission objectives?

A new study by the wonderful folks at MeriTalk, the government IT network (underwritten by NetApp), called, “The State and Local Big Data Gap,” in a word says NO! It reveals that despite the known advantages of big data, few state and local agencies are taking action to harness and analyze it.

The study is based on a survey of 150 state and local government CIOs and IT managers that take place from November to December 2012. On one hand, it found that state and local IT professionals see the value in big data. 

  • Respondents cited improving overall agency efficiency (57 percent)
  • Improving speed and accuracy of decisions (54 percent)
  • Achieving a greater understanding of citizen needs and how to meet them (37 percent) as the top advantages to tapping into big data 

Where things get problematic are on the strategy, evaluation and implementation side of things. Interest is not being translated into action, and this is not a reflection of a trickle down impact of the current challenges facing federal IT people because of the sequestration. 

While state and local IT professionals recognize the benefits of big data, few are moving on using it. The survey found that:  

  • 79 percent of state and local IT professionals say they are just somewhat or not very familiar with the term “big data”
  • Only two percent say they have a complete big data strategy
  • Big data isn’t on the radar screen for 44 percent of state and local agencies – they are not even discussing it

While they understand the promise of big data, just 59 percent of state and local agencies are analyzing the data they collect and less than half are using it to make strategic decisions. 

On average, state and local IT professionals report that it will take their agencies at least three years to take full advantage of big data

“State and local agencies have made great strides in consolidating applications and data into fewer physical resources,” said Regina Kunkle, vice president, State & Local Government, NetApp. “Storage efficiencies like de-duplication and compression help to manage the explosive storage growth by reducing the amount of storage required and simplifying data management.  However, agencies still have data silos, and they are just beginning to explore how to effectively analyze this disparate data.  To help them unlock this valuable wealth of information, agencies should look toward big data solutions.”

There is a reason it is called big data

As MeriTalk notes, “The average state and local agency stores 499 terabytes of data.  State and local IT professionals expect that amount of data to continue to grow.”

  • 87 percent of state and local agencies say the size of their stored data has grown in the last two years
  • 97 percent expect data to grow by an average of 53 percent in the next two years
  • One in three state and local agencies has a data set that has grown too large to work with given their current capacity limitations

Data challenges are daunting

The survey looked not only at the state of big data adoption but also at other challenges state and local governments are having in managing the above big waves of data they are required to manage. The results are illuminating. The challenges in order were:

  • Storage capacity (46 percent)
  • Speed of analysis/processing (34 percent)
  • Analysis (32 percent)

Where things get complicated is that respondents are unclear about who owns the data.  47 percent believe that IT owns the data and 31 percent believe ownership belongs to the department that generated it.

The bad news for vendors in the big data space is that these rather sophisticated users believe there is a gap between big data’s promise and big data reality.  The numbers are sobering. State and local agencies estimate that they have just 46 percent of the data storage/access, 42 percent of the computational power, and 35 percent of the personnel they need to successfully leverage big data.  In addition, 57 percent say their current enterprise architecture is not able to support big data initiatives.

The survey did have at least one somewhat bright spot. Despite the technology challenges, and the current funding issues which create planning challenges of their own and make new projects difficult to justify even with the ROI and TCO projects are impressive, some state and local agencies understand that employing next-generation technology is important. This is reflected in the fact that 39 percent of respondents are investing in IT systems/solutions to improve data processing, 39 percent are improving the security of stored data, and 37 percent are investing in IT infrastructure to improve data storage.

What the study reveals is that while big data is on state and local government IT professional wish lists, the more practical matters of data processing, storage and data integrity/security remain top of mind. Plus, data ownership challenges stand in the way and the business case, which must be driven by all of the stakeholders sitting at the table and agreeing to cooperate and then agreeing on a path forward, are reflected in the numbers about there being limited action and even limited discussion about big data implementations in this sector. This is unfortunate since a good case can be made that a more efficient and effective government, particularly at the levels where they are most experienced, is where big data solutions could be invaluable both in terms of service delivery and the resultant customer satisfaction. 

It will be interesting to see if these results change markedly the next time MeriTalk takes a look.    

Edited by Jamie Epstein
Related Articles

Windows 10: One Giant Step into the Future

By: Rob Enderle    7/30/2015

This is Windows 10 launch week and things are going pretty well. Unlike some rumors to the contrary, the OEMs are ready with preloads and so far, the …

Read More

Cyberattacks on OPM, Anthem and United Airlines are Linked

By: Peter Bernstein    7/30/2015

The bad news on the cyberattack front got a whole lot worse with a story from Bloomberg by reporters Michael Riley and Jordan Robertson that revealed …

Read More

Comings & Goings: Citrix CEO Steps Down as Company Gears for an Overhaul

By: Tara Seals    7/29/2015

There appears to be a bit of shakeup going on over at Citrix, the venerable distributed computing company. Long-time Citrix CEO Mark Templeton has ann…

Read More

Three Reasons Passwords Are Here to Stay

By: TMCnet Special Guest    7/28/2015

Siber Systems explains why someday passwords will be a thing of the past, but not any time soon.

Read More

Wheelings & Dealings: Visa Joins Amex with Giant Investment in Payment Startup Stripe

By: Laura Stotler    7/28/2015

Payment startup company Stripe has scored a whopping round of funding, with Visa taking a major interest in the company's technology. The funding amou…

Read More