Open Data in the EU

1. Introduction and evolution of legislation on open data

Access and re-use of data can be made difficult by public authorities due to several barriers: 

• A lack of information that specific data exists and are available; 

• A lack of clarity regarding which public authority holds the data and on the terms of re-use; 

• Data made available only in formats that are difficult or expensive to use;   

• Complicated licensing procedures or prohibitive fees; 

• Exclusive re-use agreements with one commercial entity or re-use restricted to a governmentowned company. 

• New businesses can be built on the basis of these data; 

• Transparency in public administration is increased, improving the visibility of previously inaccessible information, informing citizens and business on policies, public spending and outcomes; 

• Evidence-based policy making and administrative efficiency.

Studies conducted on behalf of the European Commission show that industries and citizens still face difficulties in finding and re-using PSI. In response, from December 2011, the Commission adopted a package of measures to overcome barriers to re-use, and to reduce the fragmentation of data markets. The key element was Directive 2013/37/EU amending Directive 2003/98/EC on the re-use of public sector information.

The Directive 2003/98/EC of the European Parliament and the Council on the re-use of public sector information (amended in 2013 and replaced by the Directive 2019/1024) aimed to facilitate the re-use of PSI throughout the EU by harmonising the primary conditions for reuse and by removing significant barriers to re-use in the internal market. It was based on two pillars of the internal market: transparency and fair competition, and it applied to all public sector bodies, who were obliged to: 

• be transparent on conditions for re-use; 
• avoid any form of discrimination between re-users;
• deal with applications for re-use within a set maximum time; 
• not enter into exclusive arrangements other than in exceptional circumstances. 

Communication from the Commission on Open Data: Open data: An engine for innovation, growth and transparent governance, 12 December 2011 

The Commission observed firstly that public data have significant and untapped potential for re-use in new products and services and that the overall economic gains resulting from the opening of this resource could amount to 40 billion Euros per year in the EU, not to mention the increased participation by citizens in political and social life. In December 2011 the Commission presented a package of measures to overcome existing barriers and fragmentation across the EU on the opening of public data which consists of three mutually reinforcing strands: 

• adapting the legal framework for data re-use, including legal, soft law and policy measures;
• mobilising financing instruments in support of open data, and deployment actions, such as the creation of European data portals;
• facilitating coordination and experience sharing across the Member States.
  

The Commission also invited the European Parliament and the Council, in their respective areas of competence, to create the correct framework conditions for the re-use of PSI within the EU and to support projects and infrastructures that can transform Europe's public data bodies into a driver for innovation, growth and transparency.

1.1. Directive 2013/37/EU amending Directive 2003/98/EC on the re-use of PSI

In July 2013 Directive 2013/37/EU amended Directive 2003/98/EC, to encourage the Member States to make as much material held by public sector bodies available for re-use as possible. The change was made necessary by the exponential increase in data and constant evolution of technologies for data analysis, exploitation and processing. 

Furthermore, due to the ambitious approaches of some Member States in implementing the 2003 Directive, which exceeded the minimum level set by the latter, minimum harmonisation was necessary to determine the type of public data available for re-use on the internal market of information, to prevent different rules adopted by different Member States from hindering the cross-border supply of products and services and to allow comparable sets of public data to be re-used to create panEuropean applications based on such data. 

The most significant changes were: 

• the expansion of the scope of the Directive to include libraries, archives, museums and university libraries (which hold a considerable amount of valuable PSI resources, particularly since digitisation projects have multiplied the amount of digital public domain material), albeit in a way that limits the possible financial effects and does not impose a significant administrative burden on these institutions;
• the obligation to allow the re-use for commercial and non-commercial purposes of generally accessible public data that are not explicitly covered by one of the exceptions;  

• data should be made available in machine-readable formats where possible (recital 21);  

• a default charging rule limited to the marginal cost of reproduction, provision and dissemination of the information. This provision does not apply to public sector bodies that are required to generate revenue to cover a substantial part of their costs relating to the performance of their public tasks (recital 22; article 6); 

• the obligation for public sector bodies to be more transparent regarding the charging rules and conditions they apply; 

• in relation to any re-use that is made of the document, public sector bodies may impose conditions, where appropriate through a licence, such as acknowledgment of source and of whether the re-user has modified the document in any way (recital 26);

• the means of redress should include the possibility of review by an impartial review body which should be organised by the constitutional and legal systems of the Member States and should not prejudge any means of redress otherwise available to applicants for re-use. It should, however, be distinct from the Member State mechanism laying down the criteria for charging above marginal costs. The means of redress should include the possibility of reviewing adverse decisions but also decisions which, although permitting re-use, could still affect applicants on other grounds, notably by the charging rules applied. The review process should be swift, due to the needs of a rapidly changing market (recital 28); 

• prohibition on cross-subsidies: if public sector bodies re-use their documents to offer added value information services in competition with other re-users, equal charges and other conditions must apply to them all. Prohibition on exclusive arrangements: public sector bodies may not enter into exclusive agreements with individual re-users, excluding others. Two exceptions apply: exclusive rights may be authorised in exceptional circumstances if they are necessary to provide services in the public interest; or in the context of digitisation of cultural resouces. 

1.2. Commission Notice

Guidelines on recommended standard licences, datasets and charging for the re-use of documents, 24 July 2014.

The purpose of the Commission Notice was  to provide a non-binding guidance on best practices within three subject areas of particular relevance for the re-use of public sector information in Europe: recommended standard licences, datasets and charging.

The guidance document was  targeted at national administrations to guide them in transposing the 2013 PSI Directive and to encourage practices that increase the role of PSI in the digital market. 

• As for licenses, the Commission recommended not imposing limits on the possibility of re-using information, but to define precise conditions for personal data protection. Re-users must be made aware of the rules on personal data processing from the beginning (for example, by including a relevant clause in the licence). Among other things, it also recommended the use of standard and already widespread licensing solutions. Finally, it specified that, in most cases, for data re-use, it is sufficient to respect the requirement of explicit citation of the source.

• With regard to datasets, the document indicated the five types of information considered of most significant interest to businesses and other potential users. These were  geographical, environmental, transport, statistical and data related to business registers and their accounting and management documents. 

• In relation to costs, the Commission asked  the public administrations to ensure that data downloading was always free or, at the very least, limited to the marginal cost of reproduction, supply and distribution.

1.3. Commission Communications

Towards a thriving Data-Driven Economy, 21 January 2015 

• availability of good quality, reliable and interoperable datasets and enabling infrastructure; 

• improved framework conditions that facilitate value generation from datasets; 

• a range of application areas where improved big data handling can make a difference.

The Commission stated that to optimise data storage, transfer, processing and analysis, the EU must use public procurement to bring the results of data technologies to the market.

Towards a common European data space, 25 April 2018 

Building on the data protection legislation in force, the Commission proposed a package of measures as a key step towards achieving a common data space in the EU, a seamless digital area on a scale that would have enabled the development of new data-based products and services. The measures put forward along with this Communication included:

• a proposal for a review of the Directive on the re-use of PSI;

• an update of the 2012 Recommendation on access to and preservation of scientific information; 

• guidance on sharing private sector data among companies and with public sector bodies for public interest purposes. 

2. Directive 2019/1024 on open data and the re-use of public sector information

As part of this package, building on the results of a public consultation, together with an extensive evaluation of the Directive and an impact assessment, the European Commission adopted a proposal for a revision of the PSI Directive, which was presented as part of a package of measures aiming to facilitate the creation of a common data space in the EU. The Proposal updated the framework setting out the conditions under which public sector data should be made available for re-use, with a particular focus on the increasing amounts of high-value data that are now available. It aimed to overcome the barriers that still prevent the full re-use of PSI, which, according to the impact assessment, included the following: 

• data generated by utilities, transport and publicly-funded research have a high re-use potential, but are not covered by the current rules, even though much of this research is wholly or partly financed by public funds; 
• providing real-time access to dynamic data held by public sector entities, for example, using APIs (Application Programming Interface), is rare, although dynamic data are one of the most commercially valid data types;
• several public sector bodies continue to charge well above what is needed to cover reproduction and dissemination costs for the re-use of public sector data. Such charges constitute a market barrier for Small and Medium-sized Enterprises (SMEs); 
• public data holders sometimes enter into arrangements with the private sector to derive extra value from their data. This creates the risk of lock-in of public sector data, benefiting large companies and thereby limiting the number of potential re-users of the data in question. 

• reduce market entry barriers, particularly for SMEs, by limiting the exceptions that allow public bodies to charge more for the re-use of their data than the marginal costs of dissemination; 
• increase the availability of data by bringing new types of public and publicly-funded data into the scope of the Directive, such as data held by public undertakings in the utilities and transport sectors and research data resulting from public funding; 
• minimise the risk of excessive first-mover advantage, which benefits large companies and thereby limits the number of potential re-users of the data in question, by requiring a more transparent process for the establishment of public–private data arrangements;
• increase business opportunities by encouraging the dissemination of dynamic data via application programming interfaces (APIs). 

• documents held by public undertakings active in the areas defined in Directive 2014/25/EU on procurement by entities operating in the water, energy, transport and postal services sectors and by public undertakings acting as public service operators under Regulation (EC) no. 1370/2007 insofar as they were produced as part of the provision of services in the general interest, as defined by law or other binding rules in the Member State;
• a specific category of documents produced as part of scientific research, namely results of the scientific fact-finding process (experiments, surveys and similar) that are at the basis of the scientific process, while publications in scientific journals continue to be excluded from the scope as they pose additional challenges in terms of rights management. 

The Directive entered into force on 16 July 2019. It replaces the Public Sector Information Directive (Directive 2003/98/EC) which dated from 2003 and was subsequently amended by the Directive 2013/37/EU.

The PSI Directive focuses on the economic aspects of the re-use of information rather than on access to information by citizens. It encourages the Member States to make as much information available for reuse as possible. It addresses material held by public sector bodies in the Member States, at national, regional and local levels, such as ministries, state agencies and municipalities, as well as organisations funded mostly by or under the control of public authorities (e.g. meteorological institutes).

Once fully transposed on the national level, the new rules will:

• stimulate the publishing of dynamic data and the uptake of Application Programme Interfaces (APIs).
• limit the exceptions which currently allow public bodies to charge more than the marginal costs of dissemination for the re-use of their data.
• enlarge the scope of the Directive to:
  • data held by public undertakings, under a specific set of rules. In principle, the Directive will only apply to data which the undertakings make available for re-use. Charges for the re-use of such data can be above marginal costs for dissemination;
  • research data resulting from public funding – Member States will be asked to develop policies for open access to publicly funded research data. New rules will also facilitate the re-usability of research data that is already contained in open repositories.
• strengthen the transparency requirements for public–private agreements involving public sector information, avoiding exclusive arrangements.

3. Open data and public procurement

At present, there is no law or policy at EU level that adequately addresses open data in Public Procurement, except for some Directives and Communications that highlight the need and importance of opening data in this sector to strengthen transparency and fight corruption. However, there are some recommendations and indications by some international organisations.

Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee, and the Committee of the Regions: Making Public Procurement work in and for Europe, 3 October 2017

This Communication presents a public procurement strategy which sets out the overall policy framework and defines clear priorities for improving procurement in practice and supporting investment within the EU. Three concrete initiatives are presented alongside this strategy: 

• communication on a mechanism for large infrastructure projects to provide clarity and guidance to public authorities on public procurement; 
• a Recommendation to professionalise public buyers as a skilled workforce is essential for effective implementation; 
• a targeted consultation on draft guidance on public procurement of innovation, to support the breakthrough of new and more sustainable solutions for our societies.

• ensuring a wider uptake of strategic public procurement; 
• professionalising public buyers; 
• improving access to procurement markets; 
• increasing transparency, integrity and better data. The Commission states that better and more accessible data on procurement should be made available as they open a wide range of opportunities to assess, on a more informed basis, the performance of procurement policies, to optimise the interaction between public procurement systems, and to shape future strategic decisions. The Commission recommends setting up publically accessible contract registers providing transparency on awarded contracts and their amendments. It also states that enabling the reporting of corruption by setting up effective reporting mechanisms and by protecting whistleblowers against retaliation can also contribute to improving the transparency of public procurement and saving public money;  
• boosting the digital transformation of procurement; 
• cooperating to procure together. 

OECD 2016 - Preventing Corruption in Public Procurement Public procurement is one of the government activities most vulnerable to corruption.

In many OECD countries, significant corruption may arise from a conflict of interest in decision-making, which may distort the allocation of resources through public procurement. The direct costs of corruption include losses of public funds through misallocations or higher expenses and lower quality of goods, services and works. In terms of indirect costs, corruption in public procurement leads to the distortion of competition, limited market access and reduced business appetite for foreign investors. Integrity risks may occur at every stage of the procurement process, from the needs assessment to the bidding phase to the contract execution and payment.

For this reason, a holistic approach to risk mitigation and corruption prevention is required.

The OECD Recommendation on Public Procurement highlights several mutually supportive principles which may, directly or indirectly, prevent corruption and stimulate good governance and accountability in public procurement. These principles include: 

• integrity; 
• transparency; 
• stakeholder participation; 
• accessibility; 
• e-procurement; 
• supervision and control.

• The 2015 OECD Recommendation on Public Procurement recommends that the adhering countries ensure a fair degree of transparency of the public procurement system in all stages of the procurement cycle. 
• The OECD Recommendation on Public Integrity supports the adherents in safeguarding integrity and public interest at all stages of the policy process, particularly by promoting transparency and open government, including actively ensuring full access to information and open data, along with effective and timely responses to requests for information.

As a minimum, adequate and timely information must be provided on upcoming contracts as well as on contract notices and information on the status of ongoing procurement processes. Additional information, such as the average procurement duration, justification of exceptions and specific overview records by type of bidding procedure, may further enable external parties to scrutinise public procurement practices. Transparency can also be further enhanced by guaranteeing the visibility of flows of public funds throughout the public financial management cycle.