California builds data warehouse to better manage homelessness
The Homeless Data Integration System will help officials identify patterns of homelessness and create a comprehensive view of the services being delivered and the interventions proving to be most effective.
According to the January 2020 point-in-time count of the nation’s homeless, over one-quarter (28%) of the people experiencing homelessness nationally are in California, while the state represents only 12% of the country’s total population.
To get a better handle on managing homelessness, California’s Homeless Coordinating and Financing Council (HCFC) developed the Homeless Data Integration System, a data warehouse that will allow the state to make data-driven policy decisions.
Built in partnership with the state’s 44 Continuums of Care, HDIS gathers and anonymizes homelessness data into a single repository. On a regular schedule, each CoC pulls data from its homeless management information system and sends it to HDIS via a secure file transfer system.
The data is cleaned and standardized. To ensure that each client has only one standardized record, HDIS uses master data management, a technique that applies a set of rules that check and validate client files by cross-matching records, flagging errors and combining duplicate entries, according to HDIS’ website.
The cleaned and matched data is then transferred to the statewide data warehouse that analysts, data scientists and policy-makers can query. A data lake at the heart of HDIS enables the development of visualizations, data modeling, statistics and analytics.
HDIS helps officials identify patterns of homelessness and create a comprehensive view of the services being delivered and the interventions proving to be most effective. It will also help the state address racial and other inequalities among people experiencing homelessness, HCFC said in its announcement.
In November, the state signed a contract with Plante Moran, one of the nation’s largest accounting, tax and consulting firms, to deliver an end-to-end system. The three-year contract called for standing up the data warehouse, obtaining and de-identifying data from 44 CoCs and creating a secure method to view the data. The company worked with a number of partners, including Informatica, Snowflake, Tableau, Amazon Web Services and the Crisis Response Network, an Arizona nonprofit that works to improve public access to local resources, according to a company announcement issued when the contract was signed.
HDIS was developed in just 15 weeks, state officials said. It can eventually be integrated other state-level data to better understand social benefits and services being accessed by people experiencing homelessness, such as CalWORKS or CalFresh.
Already, HDIS has given the state insight into several metrics that were previously unavailable, officials said. From 2017 to 2020, for example, the number of people served through homelessness prevention services increased 131.6%. In 2020, 91,626 people exited homelessness and moved into permanent housing.
"HDIS provides a foundational piece of the state's data infrastructure,” said Joy Bonaguro, the chief data officer for the California Government Operations Agency. “Before the HDIS, we had at best an infrequent and piecemeal snapshot of the homeless challenges in California. With the HDIS we will have an ongoing and holistic picture of how we can assess and improve services for those experiencing homelessness."
NEXT STORY: FCC app measures mobile broadband access