At OpenOpps, we have been working with our partners They Buy For You on an exciting new Public International Trade Opportunities (PITO) project. The outcome of this project is to provide a single point of access for data, available to people all around the world.
The first stage is to manage the data that will be used as an input for the enrichment and publication process in future stages of PITO.
We began this work in a strong position, with a database containing tender notices and contract data from across Europe, mapped to the Open Contracting Data Standard format. But we wanted to do more…
Redesigning the way we collect our data
To improve the reliability of our data collection, we have redesigned our scripts. We programmed code that collects new publications from websites publishing contracts and tenders. Now they can run independently from each other, decreasing the chances of failure across the data collection process.
In addition, this improves our ability to log the activities of our scripts. Now, for each source, we have a far better understanding of how our data collection efforts are running and performing.
Consequently, our new collection methods mean we can now offer partners an online tool to monitor script outputs every day.
Expand our data sources
Working in partnership, OpenOpps and our partners at TBFY have identified key data sources from around Europe. After assessing their suitability for the project, we created accurate scripts to collect data from these. Now we have data for Spain, Italy, the UK and France.
We have also redesigned how we process data from TED, a website dedicated to European tenders and procurement, and our contract collection scripts for TED. Following this, we can now collect even more data formats, without missing any nested contracts, multiple contracts awarded against a single tender.
At the end of this first stage of the project, we have successfully expanded the data collection required for the project. We now publish daily, on average, more than 26,000 tenders from Europe, over 200% more than our previous data gathering.
Still we want to do more. We want to keep expanding the number of data sources and to be able to better evaluate economic development opportunities from public procurement. In short, we want to keep opening up public procurement, to make it more accountable, better value for money and ultimately, fairer.
For more about Open Procurement Europe.