Wrap Up: My “digital tour of service” with the State of Colorado

Every morning I wake up to the local NPR station (KUNC) playing on my Alexa, grab a cup of coffee, lay back down and listen. It’s a preview of the weeks ahead. Working in state government puts you at the intersection of federal policy and local service delivery. Sometimes things state government builds directly impacts people, and sometimes it enables those in county and local government to serve residents and citizens in a better way.

The Colorado Digital Service (CDS), a team I co-founded with Matthew McAllister, is two years old now and I am wrapping up my “digital tour of service.” CDS is a diverse, cross-functional team of senior engineers, human-centered design specialists, product managers and procurement specialists within the Governor’s Office of Information Technology (OIT). We partner with state agencies to develop and improve human-centered solutions to Colorado’s most pressing technical challenges (and we are hiring).

Over the past two years, I’ve been incredibly proud of the work we’ve done on the COVID-19 response, with the child welfare team, and the new Paid Family and Medical Leave division. I can’t wait to see what the team delivers next.

A few lessons learned

A digital service can be an important tool for governments.

The original vision for the Colorado Digital Service was to create a small team of senior product managers, designers and engineers that would work on the Governor’s priorities and plug-in tightly with the Office of Information Technology. This has mostly played out as expected.

Why is a digital service important for your state government?
1. A digital service creates momentum for new ideas on hiring, software delivery and problem solving that can be leveraged government-wide. Publishing the Modern Software Delivery Index, creating an agile vendor pool and finding new ways to connect Colorado tech talent to government are examples of ways the Colorado Digital Service has done this here in Colorado.

2. For agency initiatives that are emerging, a digital service can be leveraged to prototype, build, ship and continuously improve upon these initiatives until the sunset or transfer into an agency program. Bug Bounty, Digital Vaccine Credentials, Exposure Notifications and Developer Evangelism are examples from the Colorado Digital Service just this year.

3. When brand new agency programs or offices are created, a digital service can help with human-centered design, rapid prototyping, product management and vendor procurements in the early stages of formation. In Colorado, we have seen a new Paid Family and Medical Leave program, Office of Early Childhood Education, and Office of Behavioral Health launch just this year.

A lot of progress has been made to improve the delivery of government services, but we still have work to do.

One of the big takeaways from the past few years in government is that some decades old rules combined with perverse incentives hamper government’s ability to deliver. Everyone wants to make government work better for the people, from tech talent who are curious about working in government, to the career civil servants that have been chipping away at problems for years.

Here are a few things that are gradually improving but need to be better.

Hiring – In Colorado, we can’t hire anyone that lives outside of Colorado, we have outdated position classifications and our product, design and engineering salaries continue to be non-competitive with the private sector.  We continue to attract great talent despite these constraints thanks to the tireless work from Colorado’s HR folks and agency directors that find creative ways to staff teams. Refactoring hiring is happening but more needs to be done such as improving how funding is allocated to these positions and competing on salary (read Tips for Finding Ways into Public Sector Work).

Delivery teams – Funding for enterprise applications can be sporadic based on a change in legislative priorities, economic climate or specifics of policy which makes it hard for government teams to hire full-time employees so outsourcing to contractors and vendors is the norm. This perpetuates a “project” instead of “product” mentality of “When will it be done?” and “We have to spend the money now.” In government, many of the technology applications that serve programs like Medicaid, child welfare and public safety are never “done”. Delivery teams need full-time government employees that can focus on delivery with a consistent funding stream over time in partnership with vendors.

Procurement – Governments can have hundreds of enterprise applications and initiatives. How government teams decide to work with contractors and vendors is a huge factor in their success. Although there has been a powerful movement in procurement reform across governments at all levels, moving to a modular, agile way of working is still new in government (watch Waldo’s talk on this). When an program team insists on developing a massive list of requirements to be delivered by a certain date for a certain price and then adds additional constraints such as a vendor having to take on substantial liability for the “system as a whole” or having a certain number of employees onsite, they think they are doing the right thing by “guaranteeing success” and reducing risk. Time after time this approach has been shown to fail (read Government tech projects fail by default. It doesn’t have to be this way).

What’s coming next for government teams is pretty easy to envision

The delivery of digital services across the government is maturing. In some Federal and State Agencies as well as local governments, you will see teams working in an agile way, using human-centered design to better understand the problems they are solving and shipping software that works on mobile devices.

Looking ahead, here are a few things that are either a growing part of a government’s portfolio or coming soon.

APIs – Interoperability between agencies, counties and community partners, open data and giving residents/citizens/patients access to their personal data is a growing piece of the puzzle for government service delivery.  We are already seeing states subject to regulation by the Federal government to provide Medicaid patients access to their health information and more of this type of regulation will come. There is also a strong demand from the private sector to access open data sets and have ways to transact with governments.

Layers of Government – For programs that span multiple agencies (CDC and CMS for example) or multiple levels of government (Federal to State to Local), a digital service can help break down silos and glue efforts together. There is already collaboration between states happening formally in monthly working groups and informally across networks of peers. Governments at all levels should be working as a portfolio in many of the large enterprise applications like unemployment insurance, Medicaid and paid family and medical leave sharing open source projects and reusing composable building blocks instead of paying the same vendors to duplicate software applications over and over for each state.

Machine Learning – Fraud detection, robotic process automation, public health outbreak prediction and so much more is coming to the government. Understanding how to manage ML pipelines, train models, and incorporate all of the ethical considerations that this technology will raise for a government into the product management process will be critical. Digital service experts that have experience training and deploying models or designing product features that leverage these models will be important. [read Digital Services to APIs to ML]

Onward!

After 20 years building software in bootstrapped small businesses, venture-backed startups and large enterprises, I feel incredibly lucky to have stumbled into government and fallen in love with the work. My work-life has never had more purpose that it does now and I’m excited to continue chipping away at these problems.

Thanks to my Colorado Digital Service teammates, Governor Polis and his team, and all of my colleagues in the Office of Information Technology and within the agencies for helping the Colorado Digital Service get off the ground and become an important part of service delivery to Coloradans. I am grateful to you for giving this idea a chance.As for me, I’m boomeranging back to the US Digital Service to serve another digital tour of service taking my learnings and inspiration from Colorado to the Federal government.

Medicaid Blue Button API Use Cases

This post is simply a list of use cases that helps all of us understand why the Medicaid Blue Button API is super important to Medicaid members.

Btw – here is the list of apps that are built using the CMS Medicare Blue Button API and another list of apps that have agreed to the CARIN code of conduct.

Providers

Claims data is one component of a patient’s complete, longitudinal digital health record adding cost, coverage and adherence.

These are ways in which Medicaid members having access to their claims information or sharing this data with their Provider can have an impact.

  • Providers can share data with each other given a patients consent
  • Patients can download their claims data into a personal health record like Apple Health and show information to their provider during the visit
  • Patients receive Infrequent procedure reminders (ex: annual exam, colonoscopy every 5 years, etc)
  • Easy way for family member/caregiver to access medical information in emergency (important for authorization access not to time out too quickly). Apple is also working on this (emerging).
  • Improving care coordination by sharing information with all members of a Medicaid member’s care team
  • Decreases printing of records a provider needs to do
  • An at-risk provider having difficulty accessing a patient’s longitudinal claims record (like we heard re: the Oncology Care Model – which shares such data up to six months after the initial trigger event) could gain more timely access to this data via Medicaid member granting permission.
  • Comparing & reconciling patient-level payor EOBs vs provider bills
  • Automatic pre-population of initial visit forms (save time, increase accuracy)

Example: Did Jane have a Cervical Cancer screening at her last OB/GYN visit?

  • Jane, a 55 yr old female Medicaid member visits her PCP. She is handed an iPad to checkin and sees “Login to your Medicaid account to share data with Dr. Smith.”
  • She logs with her Medicaid username and password
  • She sees a spinning wheel and reads “Your Medicaid claims data has been downloaded and is up-to-date!”
  • Her claims data from Medicaid is synced to Dr. Smith’s EMR
  • Dr. Smith pulls up her chart during the visit and sees that she’s had the recommended Cervical Cancer screening
  • The Provider is complying with this screening CMS quality measure

Research

From Apple’s Research Kit to Verily’s Project Baseline, making it easy for a patient to contribute health data to a clinical trial or research study is powerful.

  • A Contract Research Organization (CRO) can incorporate Medicaid claims data to improve the accuracy of monitoring during a Clinical Trial.
  • Apps like Verily’s Project Baseline enable participants to connect a variety of personal health data sources to better inform the researchers.
  • Clinical Trial enrollment – A research organization can pre-populate a medication list for a patient during clinical trial enrollment.
  • Donate data to research (ex: precision medicine, “All of Us” research cohort/Sync for Science, Health eHeart Study (UCSF)

Prescriptions

  • Medication adherence
  • Navigating affordable care options (eg brand vs generic medication)

Example: Signup for home delivery of medication 

XYZ Pharma is a full-service pharmacy that sorts your medication by the dose and delivers it to your door.  They would use Medicaid claims history to reduce the Patient’s burden during the signup process, to better understand their prior billing cadence and to build a better profile of their medication history.

Health and Fitness

Example: Personal Health Record

A 59 year old woman from Cleveland, OH named Bettie has an iPhone 8 and a Fitbit her son bought her for Christmas.  Her friend told her about an iPhone app that lets her see all of her medical records on her phone called “MedView”.  She searches for it in the app store and downloads it.  Her favorite apps on the iPhone are Weather.com and Facebook.  She opens the app and it asks her to login with her Medicaid account.  She enters her username and password and sees a “welcome message” display on the app.  Success!  The app shows her some personally identifiable information and tells her it’s downloading her entire record from Medicaid.  She sees a status bar going to 100%.  A few seconds later she sees a timeline view of her information.

Financial

  • Forecasting & planning a personal health care budget (based on previous year information)
  • Connect your Medicaid data to health insurance shopping app or an accounting app like Quicken for tax deductions.

Brainstorming list

Below is a list of terms and concepts to trigger more ideas about how patient access to claims data can help make an impact on people’s lives and health.

  • Folks moving from Medicaid to Medicare or from private health insurance on to Medicaid or vice-versa 
  • eLTSS examples
  • Home Health Care
  • Assisted Living
  • Primary Care
  • Senior Social Networks
  • Wearables
  • Transportation
  • Patient Portals
  • Chronic Disease Management
  • Social Isolation
  • Preventative Care
  • Medication Adherence
  • Digital Therapeutics 
  • EHR interoperability
  • Longitudinal Patient record
  • Patient education
  • Preference-sensitive care
  • Medication reconciliation & adherence tools
  • Care team indexing & name/ID sharing with other providers
  • Evidence-based clinical decision support
  • Quality-related application & services for Accountable Care Organizations (ACOs)
  • Chronic disease management (including personal health tracking – eg for patients with diabetes)

Good Articles

Use of the Blue Button Online Tool for Sharing Health Information: Qualitative Interviews With Patients and Providers (PubMed 2015)

New iPhone Health app feature gives doctors easier access to data (The Verge, June 2021)

Digital Services to APIs to ML

Hanging out with Brandon Williams and Lilo Santos on the Public Sector Community’s Coffee

#CivicTech is having a moment. As government gets better at delivering digital services to residents, they face challenges ahead from building APIs to wrestling with machine learning.

Digital Services

The delivery of digital services across government is maturing.  In some Federal and State Agencies as well as local governments, you will see teams working in an agile way, using human-centered design to better understand the problems they are solving and shipping software that works fine on mobile devices. Organizations like 18F, United States Digital Service, Code for America and many more are driving the conversion on building good government using technology. There is still much work to be done for government to evolve into high-performing software delivery teams and to use agile budgeting to better align team funding with how teams want to work, strengthening product management across all layers of government, modernizing legacy systems and so much more.

Open Data and APIs

We have also seen the rise of open data published by cities, counties, and state and federal agencies. This drives innovation and transparency, keys to good government.  Open datasets are typically aggregated data that do not include personal information.

Next, we will see governments making APIs available so residents, citizens, patients, taxpayers, etc can have an option to easy share the information their government holds with the apps, services, businesses and other government agencies they choose.

For example, a Medicaid patient can share their insurance claims history with a home-delivery prescription drug service or a parent can share their children’s immunization history with a summer camp that requires this information to register a child.

Offering these types of APIs will be a big challenge for most governments as they have never built information services that software developers are consuming to then provide a service to the constituent. With open data, governments mostly took a “build it and they will come” approach in which they simply made the data available and let the market figure out what to do with it. However, when building APIs on top of protected, personal information, governments will need to play an important role in helping developers successful adopt the APIs.

With API delivery, value is only realized when the developer is able to leverage the API to deliver something meaningful to the constituent.

In parallel with delivering APIs for 3rd party software developers to consume, governments will also get more sophisticated on how to exchange data within government itself. Should a State Medicaid agency be able to exchange personal health information about a patient with the State Human Services agency that offers mental health benefits that patient could leverage? Where do you draw the line and when must the constituent give their consent? These are big challenges and need people working for government that understand both the technology and business of APIs as well as policy related to data sharing.

Machine Learning

Fraud detection, robotic process automation, public health outbreak prediction and so much more is coming to government. Understanding how to manage ML pipelines, train models, and incorporate all of the ethical considerations that this technology will raise for a government into the product management process will be critical.

Books like Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor by Virginia Eubanks demonstrates how bias is introduced into systems and is super relevant for government leaders working on or thinking about building prediction or filtering systems.

Whereas vendors can offer off-the-shelf solutions like chatbots that introduce some AI into government systems without the government agency teams needing any understanding of the underlying technology, the government teams building enterprise applications will need ML engineers and experienced product managers. Government will once again need to compete with the incredible demand for this type of technologist as well as the high salaries their skills command.

If government teams still struggle with web apps, mobile experiences and stability of systems, how will they possible handle the sophistication of ML pipelines?

In government, the ROI is social impact. Residents leveraging easy-to-use digital services to pay their taxes, register a car or access a benefit helps them focus on what matters, their daily life. APIs and machine learning enable these digital services to get that much better.

We are living in an era where trust in government is low and the demand for competency and transparency in government is high. As government continues to deliver for the people, this trust improves over the long-run. We must keep working, all of us.

Consumer Access to Vaccine Records

As millions of Americans receive vaccinations in the coming months, will there be a consumer demand for vaccine data or “proof of vaccine” on their smartphones?

This NY Times article is a great read.

There is a serious debate on whether health passes, electronic vaccine credentials, proof of vaccine, etc is truly needed. Assuming it does happen, below is a quick summary of some of the puzzle pieces.

Patient Portals

When a provider administers a vaccine, it is entered into their electronic medical record system. Today, people can use apps on their smartphones to connect with the doctor’s electronic medical system called a “Patient Portal”. You setup a username and password with your doctor to login to the Patient Portal website to view your health records such as lab results, immunizations and prescriptions. 

Apps like Apple Health and CommonPass (Android) use common health data standards (HL7 FHIR) and application programming interfaces (APIs) to connect with these patient portals if your doctor has opted to make this available.

Here’s an example from Aaron Miri, CIO UT Austin showing vaccine records from a EHR connected to an iPhone displaying within the Apple Health app.

Immunization Information Systems

Each state, US territory and many large cities have an Immunization Information System (IIS) that is basically a big database that doctors, public health clinics and others have to report every vaccination into.

Vendors like Envision WebIZ and STC Health One that build these IIS systems that governments buy have consumer access portals.

Here’s an example from Nevada.

These portals use name, phone number and date or birth to match records and then the person can print out their vaccination records.

Hopefully, these vendors will introduce APIs that enable consumers connect their immunization records to their smartphones. States rely on these vendors and will likely not build these APIs on their own.

Work and School Safety Apps

There are a bunch of products that now help businesses and schools manage the health and safety of their populations. These tools will have features to track vaccinations.

An open question is whether these tools will be able to directly access vaccine records for their populations from the immunization information systems and in-turn enable consumers to have this data on their smartphones.

Tech stuff

Below is a list of additional reading on the technical topics related to interoperability of immunization data.

Medicaid Blue Button for States

States don’t need their own API sandboxes, code samples or developer support to enable 3rd party developers to adopt the new Medicaid APIs required by the CMS Interoperability and Patient Access Rule.

The rule basically says each state Medicaid plan (and many other types of plans) needs to “build their own Blue Button API” as many state health IT people say.`

I was part of the team that built the CMS Blue Button API. We had to do a few things:

  1. Map claims data to FHIR, setup a FHIR server, build an auth service, setup an API gateway
  2. Build a developer portal with a sandbox, docs, sample apps, production API access process and more.
  3. Procure the project in a way there was long-term investment in continuous improvement to the infrastructure, APIs and developer experience.

My friend Sam’s blog post Building your own Claims API from 2018 is a great read to better understand what our CMS team did and you’ll have to do for your state.

Because all states will be using the FHIR and OAuth standards, all of their APIs should be pretty much the same. This is super important because it means that states shouldn’t build their own sandboxes, docs or sample apps.

Let’s build a coalition

A group of states and organizations should get together and build an API sandbox. This will give developers a chance to play around with the APIs and prototype them into their own apps.

There should be a github site with a bunch of sample apps or build on CMS’s github along with more tutorials like Install an Angular Client App.

I’m looking forward to seeing what thought-leaders like the CARIN Alliance, CMS, vendors and states will do to bring people together here. Sharing things like a Blue Button sandbox will save taxpayer dollars and accelerate time-to-market for most states.

Happy Building!

Discoverability of COVID-19 testing sites

There is a national emphasis on COVID-19 testing now which makes the discoverability of local COVID-19 testing sites super important.

Schema.org has published new COVID-19 related tags including a CovidTestingFacility tag.

The White House has published general guidance for Federal and State CIOs

The City of Chicago’s Public Health Order 2020-6 Executive Order on April 29th mandates testing sites use these tags to publish machine-readable test site location data on their websites.

Apple Maps has these instructions on how to upload test site location data as well.

A typical discovery experience may look something like this (kinda obvious but worth noting):

  1. Person is told to get a test or thinks they need a test, searches for COVID-19 testing sites

2. After clicking “Coronavirus testing centers” they see results and a map

3. Then they click on “Website” to go to the testing location’s website

How a Civil Servant ends up building APIs

One of my favorite stories from my time in Washington, D.C. came from a dedicated civil servant who became one of my closest friends and colleagues at the Centers for Medicare and Medicaid.

With a background in public health and a passion to help people, she joined CMS 25 years ago. Over time, the policies and programs she was working on needed websites. As she took on more and more responsibility and began managing these programs, the websites now needed to allow people to login…and be discoverable on Google…and have web analytics…and conform to design guidelines…etc etc. She found herself learning about agile software development and using github.

Today, these same policies and programs now have APIs and all of the complexities that come along with building and operating APIs such as developer portals, key management, new security risks and developer evangelism to drive the adoption of these APIs.

When I asked my friend what her proudest moment has been at CMS she told me a story about helping get lead paint out of houses with kids in Baltimore. It is the impact that drives her, not the tech. She didn’t grow up coding in her parents’ basement or have fond memories of the first app she built. As Marc Andreesson says “software is eating the world” and this is so very true for government programs and the dedicated civil servants that run them.

By the way, the US Digital Service is always hiring.

Launching Your State Digital Service

A Digital Service team with a one or two year “digital tour of duty” model  is a way for the top engineers, designers and product managers in your city or state to work full-time on the civic tech problems that make a difference for your citizens.

Most of the best tech talent in your State is not working for the Government.  They have probably never considered a career in public service or have no idea where to plugin.  A Digital Service creates that pathway.

The United States Digital Service

The U.S. Digital Service was founded in 2014.  Technologists from around the country move to Washington, D.C. for a one or two year term.  Most of the 180 or so USDSers are assigned to an agency like HHS, DHS, DoD or the VA.  Some work on longer-term projects at a single agency and others will work on many varying projects across multiple agencies.  The USDS is paid for out of a part of the OMB budget called ITOR.  Because the USDS has found technology solutions that save millions of dollars and executed on projects yielding very positive results for the American people, their funding and work continues.

Setting up your State Digital Service

In a State Digital Service model, the keys are funding and air cover.  It needs support from the top, either from the Governor or Mayor.  A big decision that needs to be made is where the Digital Service lives in your State’s Government structure.  Often, the initial thinking is for the Digital Service to report into the Office of Information Technology (OIT) because “it’s a technology thing”.  If OIT works across agencies and has great leadership, this could work well.  If not, your Digital Service could simply be viewed as staff augmentation and likely will fail.

You’ll need a charter, funding and the first cohort of awesome designers, engineers and product managers from your State ready to serve.  If the right structure is put in place, the talent in your region will seek out opportunities to be involved.

Once you’re setup and have your inaugural team on board, now the real work begins.  Picking the right projects is key.

The Work

In my year with the U.S. Digital Service I have seen a few scenarios where our USDS team was uniquely positioned to make a big impact.

Cross-team Projects

At CMS, the Blue Button 2.0 project required many different teams align on a shared product vision.  As anyone that has worked in a large enterprise knows, this is very complicated and hard to pull off in the best of circumstances.  Because the USDS had the air cover from CMS leadership, we were able to recommend and execute on a path forward that brought together several teams across divisions within CMS and shipped Blue Button 2.0 within a few months.

Procurement

The USDS brings great engineers to the table alongside an agency team to assist with procurement process and technology decisions.  The USDS helped design an Agile BPA and various processes to evaluate vendor proposals that included submitting code samples to Github and more.  This is such an important part of the success of USDS, the team even has a nickname, the Procuremenati!

Design Sprints

A design sprint is a process in which a small, cross-discipline team goes really deep on a problem for a short amount of time, typically 2-4 weeks (Ex: Defense Digital Service design sprint).  The nimble structure of a Digital Service team uniquely positions it for this type of work whereas normal agency constraints may hinder a successful design sprint.

Is your State ready?

To drive innovation, find new sources of talent and generally improve the quality of the software they are building for their citizens, State and Local Governments leverage many types of structures such as University Partnerships, Bloomberg I-Team Grants, Code for America and 18F, and more.  In my home state of Colorado for example, there’s a thriving open data challenge called Go Code Colorado, a Code for Denver brigade, over 1,300 datasets in the Open Colorado data portal and learnings from work with Code for America and more.  This is the type of CivicTech energy you want as you ready for your State’s Digital Service.  It shows a strong demand to engage in Government from your local tech community.

Each team begins differently.  The US Digital Service was born out of the technical problems during the launch of healthcare.gov and is four years old now.  The Canadian Digital Service launched less than a year ago and has added GovTech leaders like Aaron Snow to the team.  The UK Digital Service began in 2010 out of a “Digital by Default” mandate and has paved the way for other Digital Service teams.

Summary

The Digital Service magic happens when you combine the private sector design, engineering, and product management talent with Government talent that has the policy knowledge, deep subject matter expertise and understands how to navigate bureaucracy.

More Reading…

Digital Service Teams

Working alongside Digital Service Teams

Playbooks and Reports

Articles

CivicTech in Colorado

I’m Joining the U.S. Digital Service

I have never been particularly interested in Government, Civic Tech and Politics with the exception of reading the Sunday NY Times, watching Meet the Press occasionally and being somewhat interested in the Open Data movement.

A year ago I had a conversation with Susannah Fox about AI, Healthcare and Aneesh Chopra’s book “The Innovative State“. I was inspired and began digging deeper into the intersection of Government and Technology. I learned about 18F, Cloud.gov and played around with datasets from data.gov. I watched hearings on the OPEN Data Act and subscribed to newsletters from Think Tanks like the Center for Data Innovation.

As it turns out, there’s an incredible amount of innovation and goodness happening in Government today from the use of open source to data transparency to progressive tech policy.

Susannah connected me with the US Digital Service, a group of Engineers, Designers, Product Managers and Digital Policy Experts that work across the Departments of Homeland Security, Veterans Affairs, Defense, Education, Health and Human Services, and the Small Business Administration to improve websites, access to datasets, user experiences and more. It sounded awesome and I decided to give it a go.

An election, a hiring freeze, a few background checks and many interviews later, I was accepted into the program. My wife, kids and I are relocating from Denver, CO to Washington, D.C. for a year long “tour of duty”.

Learn more about the US Digital Service and how you can join!

________________________________________________________

Keep Reading:

How Product Managers Can Mess Around With Open Datasets

Most Cities, States and Federal Agencies are working on some type of Open Data initiatives. The most common is an “Open Data Portal” that makes it easy to grab and use datasets:

https://data.cincinnati-oh.gov/
https://data.colorado.gov/
https://data.commerce.gov/
https://www.data.gov/

Some cities are using Open Data to publish performance metrics like the Seattle Police Department or Louisville’s LouieStat.

Civic Leaders working on these initiatives cite promoting transparency in Government, improving performance and providing data for innovation as reasons why Open Data is so important.

As a Product Manager, it’s helpful to be familiar with what’s out there and how you can play around with these datasets to better understand how your product may benefit.

Before you dive into querying APIs, checkout a few of these projects to see the end result of building something with Open Data.

USAFacts
CollegeScorecard
500 Cities Project
Data.gov

Ok, now let’s dig into some datasets you can play with.

Socrata’s Open Data Network
Socrata hosts over one hundred different data catalogs for governments, non-profits, and NGOs around the world. Checkout their Open Data Network where you can search for datasets.

For example, here’s a page about San Bernardino County Employment. Click “View API” to end up on a page giving you data and an API call you can paste into your browser or Postman.

Namara
Namara has organized a bunch of public datasets into a beautiful UI. Create a free account, sign in, create a new project then click Open Data in the left column to search and add datasets to your project. You can view the table data and manipulate it or call the data using their API.

https://api.namara.io/v0/data_sets/{DATA_SET_ID}/data/{VERSION_ID}?api_key={YOUR_API_KEY}

In your project settings, you can generate an API key. Then, in each dataset you can click “API Info” and get the data_set_id and version_id.

ProPublica
You can use ProPublica to request data about Congress such as a list of Recent Bills and Member Voting records.

https://propublica.github.io/congress-api-docs/#congress-api-documentation

You’ll need to request an API key by emailing apihelp@propublica.org then pass that in the X-API-KEY header.

For example, to query Rep. Jared Polis’s voting record:
https://api.propublica.org/congress/v1/members/P000598/votes.json

Open Data and the big IaaS Platforms

Another approach is to checkout Public Datasets baked into AWS, Google Cloud Platform and IBM Bluemix.

This is a great example of using Google BigQuery on NYC Public Datasets.

AWS hosts a bunch of Open Data in S3 buckets.

IBM, as part of the NOAA Big Data Project, has built an easy way to download tons of data.

Additional Reading

A few hashags to search around on are #govtech, #opendata, #opengovdataand #opengov. Follow people like @Josh_A_New, @JoshData, @DataInnovation, and the @SunFoundation.

Here are a few links related to Open Data policy and relevant news.

Some history on U.S. Federal Open Data Policy

DATA Act passed in 2014, America’s first open data law. It directs the federal government to transform all spending information into open data.

Conversation on the future of Open Data as Administrations change and the Preserving Government Data Act of 2017

The OPEN Government Data Act “directs all federal agencies to publish their information as machine-readable data, using searchable, open formats and requires every agency to maintain a centralized Enterprise Data Inventory that lists all data sets, and also mandates a centralized inventory for the whole government (data.gov)”.

Open Data 500 US is an interesting survey results showing what kinds of companies use which agencies’ data.