- Social impact
- Economic value: indicators
- Economic value: Charging models
- Appendix: OAIC draft survey
Communicating information to the public is a major tool and activity of government. It is how government reports what it is doing, forecasts developments in national affairs, shares research with business and the community, informs citizens of their entitlements and complaint options, and facilitates public participation in government decision making and priority setting.
How effective is this activity? Has sharing of information by government with the community produced measurable economic, social and democratic benefits? Can government information be shared more efficiently? And how do we measure those processes?
We are not able to answer those questions, but we should be asking them. The place to start is to treat government information as a national resource that should be valued, not just in theory but in a practical manner, applying a settled methodology.
This Issues Paper discusses that challenge, principally by drawing on existing research in Australia and abroad. The research highlights the boundless scope of the challenge, but also underscores its importance. Government agencies and researchers around the world are wrestling with the need to develop a methodology for valuing public sector information. Australia is advanced in publishing public sector information, and can equally be at the forefront in valuing that activity.
This Issues Paper builds on earlier work by the Office of the Australian Information Commissioner, in another Issues Paper Towards an Australian Government Information Policy (2010) and in the Principles on open public sector information (2011). We have been greatly assisted in preparing this latest Issues Paper by ideas and consultation provided by many other agencies and researchers in Australia.
To take this project forward we need a similar response to the ideas presented in this paper. We equally need cooperation by Australia Government agencies in completing the draft survey proposed in this paper.
We invite you to join us in building a better understanding of a critical national resource - public sector information.
Prof John McMillan
Australian Information Commissioner
Information is a valuable resource. The right information at the right time can expand knowledge, enable innovation, boost productivity, and even save lives. Unlike other valuable resources information is not diminished by use. Indeed, the value of information can be enhanced when it is openly accessible and reused frequently.
The term 'public sector information' appropriately describes the information gathered by government and captures its intrinsic value. In essence, it is a national resource and there are corresponding obligations on government officers to ensure that it is managed responsibly and astutely.
A key objective in government information management must be that public sector information (PSI) is made available to the community as openly as possible and is both discoverable and reusable. This was the central message in the 2009 report of the Government 2.0 Taskforce. The taskforce envisaged a role for the Office of the Australian Information Commissioner (OAIC) in promoting open PSI and recommended that the office develop a methodology for agencies to report annually to government on the value generated by published PSI.
This paper commences that process. The first step is to build a better picture of the PSI landscape in Australia. Initially, agencies must be supported in gathering the necessary data, so that a methodology for evaluating that data can be applied.
The major proposal of this Issues Paper is that agencies should complete a survey form that will gather consistent information across government on information management practices. A draft survey form is published in the Appendix. It is tied to the eight Principles on open public sector information that the OAIC launched in May 2011. The Principles reflect the information lifecycle in government information management and, as the title conveys, work from a premise of open PSI.
The paper proposes that the survey be administered by the OAIC in May 2012, following public consultation on the ideas presented in this paper. The survey will be administered both to government agencies and to re-users of PSI. The information collected in the survey will then be used by the OAIC in at least three ways. One will be in a public report by the OAIC on the Australian PSI landscape and the value of publication. The second will be in developing a methodology for valuing PSI, as envisaged by the Gov 2.0 taskforce. The third will be in discussion between the OAIC and the newly-appointed Information Advisory Council, in developing advice to government on national information policy.
The OAIC welcomes submissions by 31 January 2012 on three issues:
1.1 The ideas presented in this Issues Paper: does the paper propose a workable approach for mapping the PSI landscape and developing a methodology for valuing PSI?
1.2 The draft survey form in the Appendix to this paper: is the survey form appropriately framed to address the right issues and gather useful information?
1.3 The literature survey in Part 4 of the paper: does this survey adequately cover the field, and are their gaps or limitations in the existing research?
Open access to government information is accepted as a central plank of open government initiatives across the world. There is consensus that Public Sector Information (PSI), the information gathered and generated by public bodies, is useful to groups outside government and has considerable social and economic potential.
Governments in many countries are taking steps to increase public availability of their information holdings. At least 16 nations have undertaken major initiatives of this kind, frequently in the form of websites that make datasets available to researchers and developers. This is in addition to publication initiatives by state and local governments. Frequently, these moves to make PSI more open form part of broader-reaching transparency programs and are supported by Declarations of Open Government.
In Australia, the Government 2.0 Taskforce was created in June 2009 to advise the Government on 'structural barriers that prevent, and policies to promote, greater information disclosure, digital innovation and online engagement'.Engage: Getting on with Government 2.0, the Gov 2.0 Taskforce report, was published in December 2009 and made recommendations for making PSI in Australia more available. It argued that 'when information is released it creates new and powerful dynamics which can drive innovative use and reuse, allowing the commercial, research and community sectors to add value to it'. Since then, progress has been made in the publication of Australia's PSI; by September 2011 the site data.gov.au was home to more than 700 datasets.
The Office of the Australian Information Commissioner (OAIC) was established on 1 November 2010 with responsibilities in freedom of information (FOI), privacy and information policy. The first OAIC Issues Paper, Towards an Australian Government Information Policy, drew together Australian reports and developments that advance Australian Government information policy and proposed ten draft Principles on Open Public Sector Information (the PSI Principles). Following public consultation the ten PSI Principles were refined to a set of eight that form part of a core vision for government information management. They set out the central values of open PSI: information should be accessible without charge, based on open standards, easily discoverable, understandable, machine-readable and freely reusable and transformable. The PSI Principles are intended to help agencies embed strong information management practices into the whole information life-cycle and become confident and proactive publishers of information.
One of the recommendations of the Gov 2.0 Taskforce was that the OAIC, in conjunction with the Australian Government Information Management Office, should develop a methodology for reporting to government on the social and economic value generated from published PSI. The Government accepted this recommendation in principle, directing the OAIC 'to identify options for implementation which will further the Australian Government's information management policy and complement Australia's new FOI framework'.
The reformed Freedom of Information Act 1982 (Cth) (FOI Act) has among its objects to give the Australian community access to information held by the Government of the Commonwealth or the Government of Norfolk Island, to promote Australia's representative democracy by increasing public participation in Government processes, and to increase recognition that information held by the Government is to be managed for public purposes and is a national resource.
The FOI Act establishes an Information Publication Scheme (IPS), which requires agencies subject to the FOI Act to publish information about their structure, functions, appointments, annual reports, consultation arrangements, and details of the agency's FOI officer. In addition, agencies must also publish information routinely released in response to FOI requests and information routinely provided to parliament. Agencies are also encouraged to publish information outside the minimum requirements of the IPS and may publish exempt information if they choose. The IPS commenced in May 2011 and the FOI Act requires each agency to complete a review of its IPS compliance by 1 May 2016 in conjunction with the Information Commissioner. The OAIC will work with agencies to assist them to prepare for and to undertake a review of the operation of the IPS within their agency.
Greater openness is the central objective of the reformed FOI Act. Before information can be reused, it must be open to the public. This means it must be discoverable, in an appropriate format and under a licence that allows reuse.
In undertaking this project, the OAIC has conducted a review of similar initiatives in other jurisdictions. These are discussed in section 4. One of the central themes to emerge from these studies was the need for quality base data. Corbin's 2009 Review of indicators used in PSI studies noted that agencies do not always have good information available about their publication activities and the market for their PSI. In order to begin to assess the role and value of PSI in Australia, the OAIC considers that it is essential to understand the PSI landscape, including the extent to which PSI is open. In designing a draft methodology for measuring and valuing open PSI, the OAIC has considered the need to gather this basic data and assist agencies to begin collecting the information that will inform ongoing assessments of the social and economic value of PSI.
In addition to FOI and information management policy functions, the OAIC has responsibility for regulation under the Privacy Act 1988 (Cth) and is committed to protecting personal information from inappropriate disclosure.
PSI can contain the personal details of individuals or sufficient information to make individuals readily identifiable. As more information is published, the opportunities for data to be aggregated and personal information revealed increase. While high-level data may be less attractive to re-users due to the loss of granularity, protecting the personal information of individuals remains of great importance. Dr Kieron O'Hara of the University of Southampton has noted the potential impact of publication of PSI on privacy. He has urged greater conversation between transparency activists, privacy activists, technical experts and information entrepreneurs to ensure that the privacy-invasive potential of open PSI is minimised.
There can also be social costs accompanying open PSI. The explanatory document that accompanies the University at Albany's Public Value Assessment Tool notes the potential drawbacks of open government, including that increased availability of government information may undermine public respect for government officials.
On the other hand, advocates of open PSI stress its potential to inform the community, enabling individuals to make better decisions. A commenter on an OAIC blog said of reluctance to publish PSI, 'the intention is not evil, there are always good justifications for not releasing – it might alarm the public/ harm the government/ be misused by fringe groups/ damage national security etc.... We have to avoid the 'parent trap'... in my view the government's role isn't to decide whether the public are 'old enough' to be told certain things. Its role is to provide clear and factual information to citizens to support them in making their own decisions'.
The FOI Act now declares that the potential of information to embarrass the government is not a public interest reason to withhold access to it. However, a relevant concern is the fact that individuals may suffer material harm if certain information is made available or is made available in ways that advantage one group over another. An illustrative example is a study of the effects of publishing land-titles information in Bangalore, India, which found that digitisation of records led to more corruption, not less, and enabled wealthy land investors to exploit uncertainties in land title to build up greater holdings at the expense of poorer land-holders who did not have the same access to and facility with electronic records. The authors of the report note 'the need to replace politically neutered concepts like "transparency", "efficiency", "governance", and "best practice" [with] conceptually more rigorous terms that reflect the uneven terrain of power and control'.
The acknowledged power of information means that it must be managed with care and issues such as accessibility and social equity given due weight in information management strategies. Engagement with stakeholders and consideration of the varying impact that PSI publication may have on different sectors of the community is an essential part of the information management process. The OAIC is committed to seeing PSI in Australia managed as a national resource.
The term 'public sector information' covers a vast quantity and variety of information holdings. A methodology for valuing PSI must be capable of reflecting this diversity.
The question of objective must first be addressed, before the coverage of a PSI methodology can be settled. The Gov 2.0 Taskforce recommended that the OAIC develop a methodology 'to inform government on the social and economic value generated from published PSI'. The Government response stipulated that the methodology should 'further the Australian Government's information management policy and complement Australia's new FOI framework'.
The OAIC proposes that the methodology should provide a means of evaluating agencies' information management practices, in particular the extent to which agencies are moving towards providing open access to their PSI. Once the extent of open access is understood, the methodology can begin to assess its impact. The OAIC's view is that this initial information gathering is necessary before consideration can be given to the more complex questions of impact and value.
Understanding the PSI landscape in Australia is in itself a complex challenge. The studies on the reuse of PSI discussed in section four below found that it was difficult to obtain reliable, comparable data on the PSI market as agencies did not always have information on the sale and reuse of their PSI readily available.
An even more basic challenge lies in understanding the conditions in which PSI is reused. The study Measuring European Public Sector Information Resources (MEPSIR) identified five conditions necessary for the reuse of PSI. The first is availability: if information is not available, it cannot be reused. The next conditions are accessibility and transparency: information must be accessible to re-users and have clear reuse rights attached. Finally, the study argues that there must be accountability and non-discrimination in the PSI market. The first three of these criteria can be related to principles one, five and six of the OAIC's PSI Principles.
The OAIC suggests that a two-staged approach may be most effective in meeting the Gov 2.0 Taskforce recommendation and the Government response. The first phase would focus on gathering and analysing base data on agencies' information management policies, with a focus on the extent to which PSI in Australia may be described as 'open'. Openness is the fundamental condition for reuse. As part of assessing 'openness', the first stage would gather information on the extent to which agencies are publishing PSI in formats and under licences that support its reuse. It will also seek information about the extent to which culture inhibits or enables openness.
The information gathered in this first phase would support future studies to track changes in openness and assess the effectiveness of policy initiatives designed to increase openness. Agencies could also begin at this early stage to gather the information on reuse that would inform later study of the value generated from PSI publication -the second phase of the project. That second phase is not explored further in this paper.
It is important to start with a well-defined concept of what information is covered. The OECD defines PSI as 'information, including information products and services, generated, created, collected, processed, preserved, maintained, disseminated, or funded by or for the Government or public institution'.
The OAIC favours this broad definition, while noting that restrictions may need to apply when licensing information for community reuse, particularly cultural works over which there may be ownership rights. It is important to consider the implications of accepting this broad definition for a project such as this. Datasets, maps, photographs and research reports are each very different information products. Some of the basic tests of openness, such as discoverability and reuse rights, can be applied to most information products. However, the appropriateness of assessing economic value varies considerably. The economic value of some types, such as geospatial information, can be assessed quite readily. It is less appropriate to attempt to attach a dollar-value to cultural collections.
Indeed, consideration must be given to whether cultural works ought to be included in the methodology. During the consultation on the PSI Principles, the National Film and Sound Archive expressed concern about their holdings being subject to the Principles, especially in matters such as licensing and charging for access. Some cultural institutions are exempt from the FOI Act under s 13(1). Nevertheless, some significant initiatives have been undertaken in Australia to publish the holdings of cultural institutions and it would be desirable to consider their impact.
Publication of PSI is intended to increase access to and use of it by government, business and the wider community. Tracking the outcomes from access to data and quantifying the benefits that flow from reuse is, however, problematic. Engage notes that 'the revenue benefits from taxation of increased economic activity arising from open PSI take time to materialise'. It also concedes that 'the economic benefits ... may be diffuse and serendipitous – and so difficult to foresee'. Similarly, agencies may never know of the creative uses of their PSI or the social benefits that result from publication unless they are directly informed by the re-users of the new applications that have been found for their PSI.
The Australian National Data Service recently commissioned a report on the costs and benefits of data provision in Australia. The report contains a 'guide to data requirements' that outlines the basic data needed to understand the economics of data supply. It is necessary for agencies to get a full costing of all PSI related activities and to understand usage trends. This data is complex and the report notes that it is unlikely that any agency will have all necessary data for a complete analysis. The first phase of the OAIC survey will gather information on the mechanisms that agencies have in place to collect this data. Subsequent iterations of the survey will encourage agencies to begin collecting the data required for value analysis.
When it turns to the social value of PSI, Engage notes that 'many of the social benefits derived from PSI are not easily quantifiable in economic terms but they improve quality of life in myriad ways'. It cites the cases of the 'Mapping our Anzacs' project, which led to members of the community adding privately held materials to the national archive of service in WWI, and the National Library of Australia's historic newspaper digitisation project, which has built an enthusiastic community of individuals around the world to correct the scanned texts. This social value is difficult to quantify and must be assessed for each project individually. For this reason, the OAIC proposes that the methodology should allow agencies to submit case-studies that demonstrate the impact that open access to PSI has had.
Encouraging reuse of PSI is a goal in itself but making PSI open is also linked to the open government agenda. Increasing transparency and trust in government can be included in the potential social value of open PSI: in short, 'supporting open government' can be considered as a distinct area of value. Open PSI has the potential to support goals such as improving government accountability and efficiency and encouraging citizen participation in government decision making. Open PSI also has value to government itself. Improved information sharing within government can enhance the evidence base for decision making and reduce duplication of information collection activities. Value may be found in many areas and can be complex to quantify.
The preceding discussion is intended to give some sense of the breadth of possibilities for valuing PSI. It remains essential, however, to establish a strong understanding of the PSI landscape in Australia and the challenges currently facing agencies in their publication of PSI before attempting to assess the social and economic value generated by publication. Additionally, agencies must be supported to collect the necessary data about access to and reuse of PSI if the study is to produce meaningful results.
The OAIC proposes an initial survey to provide a snapshot of the PSI landscape in Australia. As agencies move to a presumption of openness and become more confident publishing PSI, the methodology can be developed to gather more detailed information and assess the value generated by publication.
The inaugural Information Advisory Committee (IAC) was appointed in September 2011. The IAC will provide advice to the Information Commissioner and assist in defining the scope of a national information policy. The proposed survey will highlight strengths and weaknesses in current information management practices, which will be a valuable resource for the IAC in providing this strategic advice.
A draft survey form for this first stage of the process is published in an Appendix to this paper.
This section discusses the issues and options that arise in devising a survey to be completed by agencies and re-users of PSI. The purpose of the survey is to gather information on current open PSI practices in agencies, including information about agency monitoring of data collections, publication and reuse. As indicated above, understanding the PSI landscape in Australia is an essential first step to assessing the value and impact of open access to PSI.
The first-stage survey form aims to gather that base data. Subsequent iterations of the survey will assess changes in openness and collect the more detailed data necessary to measure value. Progressive development of a survey methodology will assist agencies in collecting the necessary data and reduce the reporting burden. The information gathered will, over time, address the Gov 2.0 Taskforce recommendation that agencies and the OAIC report annually on the value generated by the publication of PSI.
The studies examined in section 4 below note that sourcing consistent and comparable data on agency activities is a significant challenge. All the studies considered have depended on self-reporting by agencies and, in some cases, private sector re-users of PSI. It appears that survey is the most feasible means of gaining the necessary information. If assessment is to be an iterative process, in order to track developments in the field, the methodology will need to be straight-forward for agencies to use.
However, the agency perspective may not alone be adequate. A methodology that sources its data exclusively from agencies risks failing to reflect the needs and experiences of PSI re-users. To that end, the sample survey form presented below contains questions on the user perspective. The OAIC will be inviting users of PSI to complete these questions and provide feedback on agencies' performance in making their PSI open.
The studies discussed below in section 4 on 'social impact' concur that it is simpler and, in some contexts, more appropriate to quantify the output of an initiative than it is to quantify the impact, especially in a complex environment.
The Public Value Assessment Tool developed by the Center for Technology in Government quantifies impact on a scale from -2 to +2, which reflects the difficulty of measuring intangibles. Furthermore, the divergent approaches taken in the studies on 'economic value' discussed in section 4 indicate the difficulty of attributing value to a single component of an initiative or product.
Those points are taken up in the draft survey form, which focuses on outputs that can be quantified. This paper has already noted the need for good base data on the PSI landscape in Australia. Additionally, this approach better reflects the range of PSI holdings and publication initiatives in Australia, rather than focussing on a few high-value types at the expense of others. Once this base data has been gathered, subsequent surveys can begin to measure change in information management practices and assess the impact of open access.
In order for PSI to be reused and generate value for government and the community it must be open; that is, available under conditions and in formats that facilitate its reuse. The starting point for a methodology that assesses the state of PSI policy in Australia should, therefore, be the openness of PSI.
Systems for assessing the openness of information have already been developed. Sir Tim Berners-Lee formulated a 5-star rating system to assess linked open data. His schema is as follows:
Available on the web (whatever format), but with an open licence
Available as machine-readable structured data (e.g. excel instead of image scan of a table)
Use a non-proprietary format (e.g. CSV instead of excel)
Use open standards from W3C to identify things, so that people can point at your stuff
All the above, plus: link your data to other people's data to provide context.
This system focuses narrowly on linked data and would not be useful for assessing the full range of PSI, which includes written reports and images. The MEPSIR study, discussed above and in greater detail in section four, identified five framework conditions necessary for reuse. These are:
- Availability: If there is not at least some information available for reuse in a particular subdomain, there will not be a market.
- Accessibility: The availability of information, per se, does not lead to anything if it is not accessible.
- Transparency: In turn, accessibility does not mean much without transparency, that is, it should be clear which conditions apply to the reuse of information.
- Accountability: The suppliers of information should be accountable for adhering to these conditions.
- Non-discrimination: Ultimately, the possibility to keep suppliers responsible for applying the same conditions to all users should guarantee non-discrimination among users.
The first three of these conditions relate directly to the openness of information while the latter two are more concerned with practices of agencies. As noted above, the first three conditions correspond to principles one, five and six of the OAIC's PSI Principles.
The OAIC launched its eight Principles on open public sector information in May 2011. They were designed to encourage agencies to embed sound information management practices, including a presumption of openness, into their internal policies and procedures. The Principles address internal practices such as staff training and compliance with legal obligations but also provide direction on matters such as format and licensing that are required for open access.
As this paper has already indicated, it is important to get a strong understanding of agencies' current publishing practices while recognising that many agencies may not have been collecting detailed information about their publishing initiatives. For this reason, the draft survey has been designed to enable agencies to describe their activities without requiring metrics that many may not be able to supply. Subsequent iterations will be able to seek more detailed information and move towards assessing value.
The OAIC draft survey is loosely tied to the PSI Principles. Given that openness is a necessary precondition for reuse, this approach will support Australia's FOI framework by enabling an assessment to be undertaken over time of agencies' progress in making their information holdings available to the community. The information gathered will indicate how easy PSI is to access and whether it is presented in ways that facilitate its reuse.
In the draft survey form published in an Appendix to this paper, questions are arranged by PSI Principle and are divided between questions for agencies and questions for re-users.
Questions for agencies are divided into 'core' and 'supplementary' questions. The OAIC recognises that agencies hold different types of PSI, which will have different application and value to re-users. The OAIC would expect all agencies to answer the core questions. The questions under the 'supplementary' heading are not applicable to all agencies and are targeted at those with substantial holdings of high-value PSI such as spatial, meteorological and statistical data. The OAIC would request that agencies do not regard the supplementary questions as merely 'optional' and answer as many of the supplementary questions as are applicable to their PSI holdings and publication activities.
Questions to PSI re-users will be administered separately in order to give balance to the perspective provided by agencies. The OAIC would ask that, as far as possible, re-users specify which agencies they source their data from and indicate how each is performing. The OAIC invites comment on both the approach taken and the questions used.
As part of the international movement to make PSI more open and available, an accompanying body of literature has developed. Studies have been undertaken on the best means of valuing PSI and the contribution that it makes to the economy, and on assessing the broader impact of initiatives in the field of open government.
The following chapter presents a selection of reports on assessing the social and economic value of PSI. Given the number of studies that have appeared in the last decade, this section is necessarily selective. The following reports have been chosen as they represent a range of approaches that have been taken in other jurisdictions. It is important to note that the studies, especially those in the section on social impact, have different goals and hence pursue differing methodologies. The relevance of the methodologies to the current challenge is therefore also variable.
This section contains summaries of four studies which have been selected for their different emphases.
The first, 'Solving social problems and demonstrating impact: A tale of two typologies', discusses approaches to demonstrating social impact and has been included as it makes the important point that it is difficult to quantify impact in complex environments. The paper suggests that, often, it may be more appropriate to focus on the outputs generated by a program.
The next two papers present tools for assessing impact. One is a Portfolio Assessment Tool created by the University at Albany. It is designed to assist agencies in demonstrating the impact of their open government initiatives. The other, the eGovernment Economics Project Measurement Framework, is designed to assess the changes effected by eGovernment initiatives.
Open PSI supports open government and is one type of eGovernment project. As such, both these tools have elements that are relevant to the current challenge but neither provides an exact 'fit'. The first tool is focussed on assessing public value, which it breaks down into a number of categories. This approach is useful as 'value' is, as discussed above, a broad concept and often ill-defined. The eGovernment Measurement Framework is designed to measure the impact of eGovernment initiatives, which includes a greater range of projects than publishing PSI.
The fourth paper in this section uses the Measurement Framework to assess the impact of an initiative to publish spatial data in Catalonia. It demonstrates the usefulness of the framework and highlights some of its limitations.
This section also describes the methodology used by the US Government to assess the progress of agencies towards fulfilling the requirements of the Open Government Directive, issued in 2009.
This paper addresses the challenges faced by the not-for-profit sector and social enterprises in demonstrating that programs addressing entrenched social problems have achieved the desired outcomes.
Gianni Zappalà notes that 'the value generated by social programs is not always immediately or clearly observable, as it is often tacit and embodied in processes rather than explicit in particular outcomes'. While the paper is primarily concerned with the challenges faced in demonstrating the effectiveness of programmes designed to tackle social problems, the challenge of measuring the value or impact of an initiative in a complex environment is similar to the challenge of identifying and quantifying the social and economic value of published PSI.
Zappalà discusses a framework for measuring social impact which could be applied to the current challenge of identifying the value generated from published PSI. This framework seeks to identify the most appropriate ways of measuring the impact of a given strategy, arguing that the effectiveness of 'focused' initiatives, such as the provision of emergency services, should not be measured in the same way as initiatives aiming for 'ecosystem results', such as resource management programmes.
The framework assesses projects according to whether the theory of change and the operation strategy employed are 'focused' or 'complex'. According to Zappalà, 'programs with a complex Theory of Change are those where the relationship between cause and effect is non-linear and there are multiple factors at play and the patterns of causation are not clear'. The second dimension of the framework concerns the operational strategy, which may either be highly specific and focused or may consist of a number of activities.
The social and economic impact of the publication of PSI and its reuse would fall into the 'complex' category regarding the Theory of Change, as the environment is complex with many factors in play. The operational strategy could be characterised as either focused or complex. The 'focused' aspect of the strategy is to publish PSI in ways that will maximise its useability. According to the framework, the results of a focused strategy such as this are best measured in terms of 'outputs and influence'.
The role of Open PSI in increasing innovation and building citizen engagement with government would be characterised as part of a complex operational strategy, which produces 'ecosystem results'. According to the Framework, 'ecosystem results' such as economic and collaborative development are best measured in terms of outcomes and impacts. These are the hardest to quantify, as the complexity of the environment means that it is difficult to attribute outcomes to a policy or program with confidence.
This paper suggests that there are limited circumstances in which it will be appropriate or practicable for an organisation to attempt to measure the impact of its programs. For the present project, it may be more useful to concentrate on the 'focused strategy' of publishing PSI and hence on quantifying the outputs that the publication policy produces rather than attempting to measure impact.
The Center for Technology in Government at the University at Albany devised a tool in consultation with a number of US government agencies to assist them 'to better understand and articulate the value of open government efforts to a range of stakeholder and interests'. Seven federal agencies participated in the initial workshop phase and two trialled the tool and provided further feedback. The tool itself is available under a free licence for use by government and is accompanied by a report that explains the rationale behind it.
The basic purpose of the tool is to help agencies to understand the public value of open government initiatives. The scope of the tool is therefore broader than measuring the value of published PSI, although giving open access to PSI forms part of the open government agenda.
The tool identifies a range of Public Value Impacts, which are described as 'categories that capture the range of possible results of opening government'. The Value Impacts are:
- Economic – impacts on current or future income, asset values, liabilities entitlements, or other aspects of wealth or risks to any of the above
- Political – impacts on a person's or group's influence on government actions or policy, on their role in political affairs, influence in political parties, or prospects for public office
- Social – impacts on family or community relationships, social mobility, status and identity
- Strategic – impacts on a person's or group's economic or political advantage or opportunities, goals and resources for innovation or planning
- Quality of life – impacts on individual and household health, security, satisfaction, and general well-being
- Ideological – impacts on beliefs, moral or ethical commitments, alignment of government actions or policies or social outcomes with beliefs, moral, or ethical positions.
- Stewardship – impacts on the public's view of government officials as faithful stewards or guardians of the value of the government in terms of public trust, integrity, and legitimacy.
From identifying these 'value impacts', the tool then considers the meaning of 'value' according to four criteria: Value in what sense? For whom? By what mechanisms? And under what conditions? It proposes an answer in terms of six changes that may be achieved by openness:
- Efficiency – changes in outputs or goal attainment with the same resources, or obtaining the same outputs or goals with lower resource consumption
- Effectiveness – changes in the quality and/ or quantity of the desired outcome
- Intrinsic enhancements – changing the environment or circumstances of a stakeholder in ways that are valued for their own sake
- Transparency – change in access to information about the actions of government officials or operation of government programs that enhances accountability or citizen influence on government
- Participation – changes in frequency and intensity of direct citizen involvement in decision making about or operation of government programs or in selection of or actions of officials
- Collaboration – changes in frequency or duration of activities in which more than one set of stakeholders share responsibility or authority for decisions about operation, policies, or actions of government.
As the tool is designed to measure the impact of open government initiatives, the range of value impacts that it proposes is broader than may be appropriate for a project concerned more narrowly with measuring the value of PSI. Additionally, the tool is designed to assist agencies to evaluate a suite of open government initiatives, rather than a single project or cluster of projects such as PSI publishing. However, the range of impacts and changes proposed offers insight into the value of PSI beyond the economic. The methods for measuring value, such as changes in efficiency, effectiveness and participation may be useful for gauging value beyond dollar value.
The tool itself is available under a variety of licences, including a free licence for use by government agencies. Using the tool itself is a six-step process, which may be complex to administer, especially for agencies with a number of large projects.
The eGEP Measurement Framework was developed to measure the impact of eGovernment initiatives and considers the three 'value drivers' of efficiency, democracy and effectiveness. It draws on five European national methodologies to produce a list of 92 'impact indicators' that assess the value that eGovernment initiatives may generate, in both quantitative (financial) and qualitative terms.
The paper notes that there is increasing interest in measuring both the social and financial impact of initiatives and that there is a recognised need for systematic and reliable measurement tools. As the Measurement Framework is designed to assess the impact of eGovernment initiatives broadly, its scope extends beyond what would be needed to measure the value generated from the publication of PSI. The authors note that the Framework is intended to provide a range of indicators from which policy makers can select the most appropriate ones for their needs.
The eGEP Framework presents means of assessing three different areas of impact: efficiency, effectiveness and democracy. The Efficiency indicators are grouped under the heads of cashable financial gains, better empowered employees and better organisational and IT architecture. Effectiveness is quantified as reduced administrative burden, increased user valuation and satisfaction and more inclusive public service. Democracy is assessed under the headings of openness, transparency and accountability, and participation.
The measures of effectiveness are most likely to be relevant to the project of valuing PSI, although better access to PSI is also likely to add value under the headings of efficiency and democracy. The Measurement Framework assesses change in each of its indicator classes, which necessitates initial benchmarking so that progress can be tracked. The authors note that this focus on change will disadvantage agencies that are already high-performing, as they will achieve less change. To balance this effect, the authors suggest scoring agencies against a 'mean' in each category in order to indicate agencies that are already performing well against each indicator.
The Framework provides means for agencies to assess the impact of their eGovernment initiatives. The eGEP Measurement Framework considers possible sources of data and assesses their relevance, cost and comparability. They suggest a variety of data sources, including web metrics, third-party assessment and user satisfaction survey, and official statistics and records. While conducting surveys would take some time and resource allocation, agencies could reasonably track most of the suggested data sources themselves, making them suitable for a self-assessment tool.
Finally, the report comments on the differences between output and impact, noting that 'it is often much more difficult to relate production processes directly to effects (outcome) than to output'. As the framework for measuring social impact discussed in Zappalà, above, notes, it is frequently more meaningful to assess the outputs of specific services or initiatives. Outcomes in eGovernment depend on outputs produced by the public sector but many other variables are also in play. In cases where the gains are not readily tangible, the Framework favours indicators that measure output or usage of services. This approach has the advantage of producing results that are ultimately more meaningful than those which attempt to attribute causality in a complex environment.
This report presents a study of users of the Spatial Data Infrastructure of Catalonia (IDEC). IDEC was established over a five-year period at a total direct cost of €1.5 million to promote the use of geographic information by improving access to it.
The study utilises the eGEP Measurement Framework. While the Measurement Framework could be used by an agency to assess the impact of a suite of initiatives, in this case it has been used to assess the impact of a single initiative, the establishment of IDEC, on users. The study focuses on two main groups of users, local authorities, and institutional and private-sector companies.
The study selected a range of indicators from the full list presented by eGEP and surveyed respondents. The authors of the report note that face-to-face interviews made it possible to explain the methodology to respondents which improved the quality of responses. The Measurement Framework calls for survey participants to identify a percentage change in each of the selected indicators.
In the Catalan study, 70% of local authorities reported saving in time and 60% a reduction in costs. However, only between 30% and 45% of respondents were able to quantify these savings. This highlights the difficulty of quantifying impact, even when specific measures are supplied.
The Framework includes indicators for assessing employee satisfaction and decision-making, and user satisfaction, some of which were utilised in this study. The survey found that IDEC's impact varied widely across agencies in these regards, suggesting that it takes time for agencies to adopt new approaches and see the benefits. It may also indicate that agencies must make a commitment to new initiatives if the full benefits are to be realised. User satisfaction could only be assessed in an anecdotal way, although 65% of internal users reported high to medium levels of satisfaction.
The study concluded that IDEC has had positive social and political effects, although there is still progress to be made in developing a 'culture of shared data'. The greatest benefits were noted by small municipalities, which are now able to access data and provide services that their limited resources previously precluded. It took four years from the launch of IDEC for concrete results to be felt but the study concluded that, in terms of the internal efficiencies gained, the investment in setting up the service was recovered in just six months. This study demonstrates the usefulness of the eGEP framework in demonstrating the benefits of initiatives but also the necessity of committing to them.
The 2009 Open Government Directive, issued by the Obama administration in the United States, called on the Chief Technology Officer and Chief Information Officer to create a tool to assess the progress of open government initiatives in the Executive Branch. A scorecard – the Open Government Dashboard – was developed that indicates the progress of agencies in a variety of open government initiatives. Based on their responses to a survey, agencies are ranked as either 'meets expectations', 'progress towards expectations' or 'fails to meet expectations'.
The tool is designed to assess progress towards the goals outlined in the Open Government Directive and judges agencies' performance in areas such as transparency, participation and public consultation. The scorecard incorporates criteria for the publication of high value data and data integrity; public consultation, participation, transparency and collaboration; and an agency's overall plan for fulfilling the directive. The dashboard gives a visual representation of agencies' progress. More detailed information is gathered by survey. It does not measure economic value but a similar design could be useful to represent the progress of agencies in publishing PSI and implementing good information management strategies generally.
The following group of papers is a selection of studies that have attempted to determine the economic value of PSI. The first is a new study undertaken in Australia for the Australian National Data Service that examines the costs and benefits of data provision. The study finds significant social and economic benefits can follow from open access.
The next study by Corbin is useful as it assembles the economic indicators utilised in earlier studies. No single set of agreed indicators emerges from this work and it is likely that, due to the diverse nature of PSI itself, there will always have to be some flexibility in the indicators used.
Three other economic studies are also discussed. One of these studies attempts to value PSI in the UK while the other two focus on the EU. Of particular note is the lack of certainty in the figures that these studies generate. They are dependent on survey and estimate for their data and, due to the uncertainty in the base data, produce results with wide ranges of values. As well as the wide value-ranges within the reports, there is also a significant difference between the conclusions reached by the two EU-based studies. Studies may either focus on the overall value of PSI-dependent sectors in the economy, which risks over-estimating the value of the component PSI, or focus more narrowly on the costs of producing PSI, which neglects the economic value generated by its reuse and may underestimate the its true economic significance.
As discussed in the introduction, research has also been undertaken into the economic impact of different charging models. These studies have concluded that the benefits, including greater reuse, that flow from free access to PSI outweigh the loss of revenue. As PSI is increasingly made available at minimal cost and on open licensing terms, the usefulness indicators related to revenue generated and licences issued will decline.
This report explores the costs and benefits of making PSI freely available to both agencies and users of data and presents some estimates of the wider economic impacts of open access. The costs and benefits are demonstrated through case studies of three data types: national statistics, spatial data, and hydrological data. The study focuses on the costs and cost savings experienced by both the PSI producing agencies and the re-users in an open access model and also considers the wider social and economic impacts of open access.
While recognising that open access to PSI can impact on agency and user costs in complex ways, the study proposes a framework for estimating cost benefits that focuses on costs and cost savings. The framework breaks down activities of producers and re-users of PSI into phases based on the information lifecycle. It then considers the possible costs of moving to free access, creative commons standard licensing and standard data formats for each phase.
The greatest cost to agencies lies in the loss of revenue when information that was previously sold is provided at marginal cost. The report argues that moving to standard licences and formats may have some transitioning costs for agencies but is unlikely to have a material impact on costs once the standard systems are in place. Ultimately, the use of standard licences and formats should reduce agency costs by reducing the support required by re-users. Free access and standard licences and formats should all reduce user costs.
In addition to the direct costs and savings to producers and users of PSI, the framework considers a range of possible efficiency and productivity impacts. The report notes that there may be some costs for early adopters and the model may underestimate the long term advantages of open access. Finally, the report notes possible wider economic benefits, such as the development of new products and services. As noted elsewhere, these benefits are very difficult to predict and quantify but may be significant.
The three case studies demonstrate the diversity of PSI and its applications. The first considers the moves towards open access by the Australian Bureau of Statistics. The study found making publications and statistics freely available online led to revenue loss of about $4.5 million in 2006–07 and generated cost savings of around $945,000. Moving to Creative Commos (CC) licensing produced further savings, resulting in an annual loss of approximately $3.5 million. The savings to users and associated efficiency gains, coupled with the increased use that the ABS noted when information was made available free, may have amounted to benefits of around $25 million per annum, or benefits of around 5.3 times the costs.
In the case of the spatial data managed by the Office of Spatial Data Management and Geoscience Australia, moving to free access caused revenue losses of approximately $1.3 million per annum, with savings of about $375,000. Users saved approximately $1.7 million per annum. The impact of increased use suggests an increase in social return of $15 million, or around 13 times the revenue foregone.
The third case study considered hydrological data managed by the National Water Commission and Bureau of Meteorology. The study considered that 'the centralised and coordinated management and open availability of water data is too recent a development for its impacts to be evident'. Water information is, however, important for crop strategies, drought management and decisions about water allocation. Making water information readily available will enable better management of this critical resource. These case studies indicate that making information freely available increases the return on the investment in that information. As well as the economic impact, free flow of PSI can have considerable social value.
This paper by Corbin was produced by the working group established by the European Commission Public Sector Information Group to review the methodologies used in studies of the value of PSI. The review considered six studies published between 2000 and 2008, including the MEPSIR, Cambridge and Pira studies discussed below.
These studies experienced a number of the same difficulties with obtaining data and the consistency of data where it was available. All were forced to rely on interview and estimating techniques. In particular, the studies found that users of PSI were reluctant to provide information about the value of the PSI to their business for reasons of commercial sensitivity or fear of upsetting their relationship with the data-agency. Additionally, the public sector bodies themselves frequently did not have a strong understanding of their customer base or the uses to which their information is put. The review noted that most of the studies did not deal with procurement of data by the public sector or the complex issue of downstream value-adding, as it is difficult to identify PSI re-users more than one or two removes from the data source. The Pira report recommended establishing a market monitor as a matter of priority to enable the valuation of the PSI market and its long-term impact on areas such as revenue growth and employment.
The studies employed a number of different approaches to their research. Four assessed the income generated by supplying PSI while three examined the cost of procuring it. Three also considered the numbers of staff employed, the financial turnover of organisations and numbers of licences granted. The focus in all of these studies was on the monetary value of PSI, rather than the social benefits. As agencies increasingly make PSI available at marginal cost and on open licensing terms, the usefulness of revenue and licences granted as indicators of value will distort. All of these indicators have weaknesses and each study considered a number of indicators in an attempt to gain a more complete picture of the PSI marketplace. The review offers little evaluation of each study or the indicators employed as it was a preliminary study prepared to support a larger project, discussed below.
In February 2010, Corbin published a blog post on the European Public Sector Information Platform Towards a set of PSI reuse economic indicators. In this post, he explained that the working group had reviewed a further nine research papers from the geographic information sector. He said:
- [T]he process so far has identified over 25 indicators of which only six have been used in three or more of the studies or research papers reviewed. The six indicators identified ranked in the order of use include:
- 1. Number of staff employed (6 occurrences)
- 2. The Organisation's financial turnover (5 occurrences)
- 3. The data supply income (4 occurrences)
- 4. The number of licences (4 occurrences)
- 5. The total income (3 occurrences)
- 6. The PSI procurement cost (3 occurrences).
Corbin noted that, due to rapid development in the field, it is possible that some important indicators have not been identified, and it remains to be seen whether the identified indicators will be easy to use in regular measurements and whether the indicators are applicable to the supply side, the demand side or both. A commenter on the blog suggested that use of CC licences has the potential to distort the 'number of licences' indicator.
Following the initial studies of economic indicators discussed above, the European Public Sector Information Working Group then assessed the usefulness of the indicators for three sectors of PSI: address information, cadastral information, and meteorological information. This sector-based approach was adopted as earlier research had indicated that the PSI market is not homogenous and suggested different indicators should be assessed for different types of PSI. The report further noted the differences among the public sector organisations surveyed, including the charging models they employed.
The study highlighted the challenge of identifying indicators that would be both stable over time and easy to measure. At the conclusion of the three studies, no common, standardised and uniform set of indicators to measure the reuse of PSI had been agreed. The report listed the ten indicators recommended by the working groups. These are:
- 1. Number of licences issued/delivered/sold
- 2. Number of online subscribers to information
- 3. Level of income generated from the supply of data
- 4. Growth rate – for example: in customers, in requests, in bandwidth.
- 5. Total income
- 6. Volume of downloaded information
- 7. Resource allocation
- 8. Financial Turnover (trend) of specific companies operating in a given thematic sector
- 9. The growth of products based on reuse of PSI in a given thematic sector
- 10. User confidence.
The recommended indicators cover both the supply (7 indicators) and demand (3 indicators) sides of the open PSI relationship. Most are considered easy to measure but do not provide a direct economic value and cover only immediate downstream uses. The work of this group indicates that measuring the economic value of PSI reuse is complex and, due to its diffuse nature, is difficult to quantify beyond first- or second-remove users.
In 2006, the UK Office of Fair Trading published a large study into the commercial uses of PSI which evaluated the value of PSI to the UK economy and considered whether PSI could be used more effectively and hence this value increased. The study relied largely on interviews for its data - the Office of Fair Trading interviewed 400 agencies (PSI Holders) and 300 businesses that use PSI. This data was then provided to DotEcon, economics consultants, who prepared a report on the commercial use of PSI.
DotEcon took a 'bottom up' approach, focussing on the direct revenue generated from the sale and licensing of PSI. They suggested that attempts to take a 'top down' approach, which focuses on the total value of sectors that use PSI as an input, tend to overestimate the value of PSI. DotEcon's modelling defined the economic value of PSI as the 'net surplus' generated by the provision of PSI, being the sum of the price that consumers would be prepared to pay over and above the current prices paid and the extent to which suppliers might be able to obtain revenues in excess of the cost of supplying the product or service. The report noted that lack of data was the main limitation to the study and that some assumptions were necessary. As well as assessing the demand for PSI, the report considered economic detriment, which is the difference between the value generated by current exploitation of PSI and the value that might be achieved if PSI were exploited more efficiently.
The report found that in 2004–05, £400 million was earned from the sale, supply and use of PSI, with a further £190 million derived from developments and reuse. These figures reflect some particular features of the PSI market in the UK. 75% of the £400 million derived from five Trading Funds, public bodies that sell their information as a commercial enterprise. Half the revenue came from purchase by business, a further 45% from public sector bodies and the remaining 5% from the general public. The report estimated that, if current barriers to efficient reuse were removed, this value could double to over £1 billion. As it is focused on the revenues generated from the sale and licensing of PSI, DotEcon's report does not engage with the contention that PSI generates greater value when it is made available at minimum cost.
This study was undertaken for the European Commission as part of the preparations for the review of the European Directive 2003/98/EC on the reuse of PSI (the Directive). The main objective of the Directive is to establish 'a minimum set of rules governing the reuse and the practical means of facilitating reuse of existing documents held by public bodies of the member states'. The two objectives of the study were 'to develop, document and test a repeatable methodology for measurement of PSI reuse and to perform a baseline measurement of PSI reuse in the European Union and Norway'.
The study investigated reuse in six sectors of PSI. The authors noted that the Directive can only have indirect impact on national markets; framework conditions must be achieved to facilitate reuse of PSI. Once these are met, economic change may begin to take place. The study was conducted by a mixture of desk research, a web survey and online questionnaires. The major component of the research was self-reporting by public content holders and PSI re-users, which included the provision of key economic data. Other studies have noted reluctance to supply this data, especially on the part of business, but the MEPSIR researchers do not mention any difficulties in obtaining data.
The report took two approaches to estimating the size of the overall market for PSI in the EU. The first approach was to ask PSI holders and re-users to estimate the size of the domestic market for the PSI sectors in which they are active. The authors noted a wide range in responses, indicating very different perceptions of each domain among PSI holders and users. PSI holders tended to report lower values than PSI re-users. Based on the estimates of the re-users, the study suggested that the overall market for PSI in the EU plus Norway is €26.1 billion, with a possible upper value of €47.8 billion.
The second approach involved the construction of proxies based on economic data (such as annual turnover and numbers of staff) that were also provided by survey respondents. While the report suggested that these figures were more robust than the estimates provided by survey respondents, the calculations still produced extremely large ranges in results. The minimum estimate for the overall market size in the EU and Norway is €10.3 billion, with an upper estimate of €44.9 billion and a mean value around €27.6 billion. The mean and upper figures are broadly in line figures produced by the estimated responses. It is worth noting that the mean values reached by the MEPSIR report are in line with the lower values estimated by the Pira report in 2000 (discussed below). This variation is indicative of the problems associated with estimating the commercial value of PSI and in particular the lack of agreed methodology.
In addition to estimating the value of the PSI market, the MEPSIR report surveyed PSI holders and re-users in order to establish benchmarks for the maturity of the market in each country according to the five framework conditions identified at the outset of the study. This survey asked both the holders and re-users of PSI questions about the availability of PSI, including licensing terms and time taken to respond to requests for information.
The results of this survey were used to rate countries' PSI policies according to accessibility, transparency, accountability and non-discrimination. Based on the survey results, the authors concluded that there was still a significant gap between the current situation and the one sought by the Directive. They suggested, however, that the Directive has the potential to affect economic performance by improving the accessibility of PSI.
This report by Pira International was produced for the European Commission and examined how commercial exploitation could help to maximise the value of PSI. It made a number of recommendations about how to increase the commercial reuse of PSI, which at the time of writing was well behind that in the US.
As part of the study, the authors investigated the present value of PSI in the fifteen member-states of the European Union. They explored various means of valuing PSI, considering both the supply and demand sides of the exchange. On the supply side, the investment costs incurred by government and the non-government sector in gathering the information represent one 'value' of PSI although the authors consider this value to be an underestimate. On the demand side, both expenditure on PSI and resources spent accessing it need to be considered. When PSI is sold or passed directly to its final users, the cost of access is a useful figure. Estimating 'final value' is, however, a more complex process as it also needs to take the value added by private-sector re-users into consideration.
Detailed data on value-added services was not available for any country so the authors had to use estimates. They note the lack of an established methodology for estimating this added value and suggest that conventional methods tend to overestimate the value of the PSI: while PSI is important to many industries, to ascribe the total economic turn-over of these industries to the PSI overestimates its value.  The authors of the report therefore attempted to extract the value of the base PSI from value-added products, while acknowledging that this approach is likely to result in an underestimate of value.
The report investigated investment in and sale of PSI across the EU and arrived at a central estimate of €68.5 billion per annum with an upper estimate of €134 billion and a lower estimate of €27.7 billion. The estimates were produced by using detailed figures where they were available (for five out of 15 member-states) and then extrapolating. The report recommended that the EU establish a 'market monitor' to monitor both the enforcement of regulatory measures and the overall success of strategies to maximise the value of PSI. This would include monitoring the tax revenue generated through enhanced PSI trading, employment growth in the information sector attributable to PSI trading and growth in the turnover and profit of the EU information trading sector.
Complementing studies estimating the economic value of PSI are investigations into the economic impact of the charging models applied to PSI. An early example of this type of study is a study by Weiss, 'Borders in Cyberspace: conflicting Public Sector Information Policies and their Economic Impacts'(2002). This study compares the impact of two approaches that states can apply: either giving access inexpensively and widely, or restricting access and managing PSI as a source of revenue.
A subsequent study in the United Kingdom, commissioned jointly by the Department for Business, Enterprise and Regulatory Reform and HM Treasury, considered the economic impact of different charging models applied by Trading Funds. This study focused on comparing the existing cost-recovery approach taken by the majority of Trading Funds with the effects that might be achieved were marginal costs charged.
In his study, Weiss suggests that the failure of information industries to develop in Europe on the same scale as they had in the US is due to the restrictive approach taken to licensing PSI taken in many European nations. He discusses several case studies of open-access and cost recovery approaches and concludes that a cost-recovery approach brings with it a number of problems. Allowing government agencies to sell both raw data (such as geographic information) and value-added products based on the same data (such as maps) can create an uncompetitive environment, which in turn discourages other commercial users of that data.
If the price of PSI is unreasonably high, users may seek the same information from other sources or find ways to operate without it. High costs for information can also restrict users to a narrow group of specialists who use it for predefined purposes, discouraging innovation. Weiss concludes that charging marginal costs for PSI will lead to economic growth by encouraging reuse of data and expansion of information-based industry. This overall economic growth, he believes, will outweigh any loss of direct revenue generated by a cost-recovery model. He does concede, however, that these 'indirect' economic benefits are not always easy to quantify and track.
The study of Trading Funds in the UK reached broadly similar conclusions. The authors noted a number of challenges that they encountered in their study. These included the limitations on the available data, especially on the demand side of the market, and ambiguities concerning the nature of the data products. The study adopted the basic approach of considering the changes in consumer surplus, producer surplus and 'total welfare' achieved by the different charging models. The authors argued that, as demand for digital data is likely to be high and there are likely to be benefits from facilitating new uses of the data, and as the government already contributes a large contribution towards fixed costs, the case for the cost-recovery model is limited. The study concluded that, in the majority of cases, the most efficient cost of supplying data is marginal cost.
The OAIC raised the issue of charging for access to PSI in the development of its PSI Principles. Comments on Principle 7, 'Appropriate charging for access', were divided. Many submissions supported access at zero or marginal cost while others noted the importance of charging to the operations of some agencies. The revised Principle 7 directs agencies to provide access to PSI at the lowest reasonable cost. While the OAIC recognises that the benefits of open PSI are often difficult to quantify in contrast to revenue generated by sale or licensing, a minimum cost approach best supports the objects of the FOI Act, which require that PSI be managed for public purposes and recognised as a national resource, by maximising reuse.
How open is your PSI?
- What information does your agency routinely publish in addition to the requirements of the IPS?
- Does your agency have a plan in place for increasing open access to its information holdings?
- What are the most significant challenges faced by your agency in publishing information?
- Does your agency have processes in place to consult the community in deciding what information to publish and about agency publication practices?
- What processes has your agency established to respond to comments received from the community and to requests for information?
- Please describe any initiatives that employ Web 2.0 tools to support community consultation.
- Does your agency work with a community of re-users? If so, please describe.
- Does your agency have channels in place for the community to provide feedback about the quality, completeness, usefulness and accuracy of published information?
- Are you satisfied with the channels in place for you to request data that you require?
- Are channels in place for you to provide feedback of the completeness and accuracy of data? Are you satisfied that feedback is acted upon?
- Who, in your agency, is responsible for providing leadership on agency compliance with the Information Publication Scheme and Disclosure Log?
- Does your agency have processes are in place for ensuring agency compliance with legislative and policy requirements on information management and publication?
- Can you describe your agency's process for ensuring compliance with legislative requirements on information management and publication?
- Does your agency maintain an asset inventory or register of the agency's information? If not, what plans exist to create one?
- How are the custodians of each information holding identified to internal and external stakeholders? Are their responsibilities clearly explained?
- Are up to date contact details for the custodian of published information available with that information?
- What staff training in information management is in place? Are there plans to develop further training?
- Are you able to readily identify the information that you require?
- Are up to date contact details of the officer responsible for that information easy to locate?
- Is information published online by your agency in an open and standards-based format?
- Does your agency attach metadata to information so that it can be easily located? If yes, what metadata standards are employed?
- What steps is your agency taking to ensure that information published online complies with the Web Content Accessibility Guidelinesversion 2 (WCAG 2.0)?
- Does your agency publish its data in appropriate repositories and/or catalogues? If so, which ones? If not, what steps is your agency taking to ensure discoverability?
- Is your agency able to track visits to its website(s) and downloads of content?
- Are there registration or sign-up processes attached to downloading or accessing data published by your agency?
- Does your agency collect statistics about access to or use of published information?
- Is data published by your agency in machine-readable formats?
- Is an update schedule in place for time-sensitive datasets? Is this schedule explained?
- What proportion of datasets is published as structured data? What proportion is published as linked data?
- Do you apply a geospatial location (e.g. physical address, latitude/longitude, administrative boundary, road/rail/river name, water body, etc) to any published datasets?
- Under what sorts of licences, including open licences, does your agency publish information?
- If your agency does not already do so, does it intend to transition to using open licences where possible?
- Do you have mechanisms in place for assessing the agency costs (including staff time) of administering licences?
- Does your agency have any exclusive agreements with re-users for particular data sets? If yes, please give details.
- What proportion of the information that you use is available under an open licence?
- Are additional restrictions applied to licences?
- What charging regime(s) does your agency apply to the information that it publishes?
- How is this regime explained to purchasers?
- How often is this regime reviewed?
- Do you have mechanisms in place for assessing the agency costs (including staff time) of administering charging?
- What percentage of published data is available free of charge?
- What percentage of published data is made available at marginal cost?
- How much income does publication generate for your agency?
- Is the charging regime adequately explained?
- Are there exclusive agreements in place?
- Is the procedure for enquiries and complaints published? If so, where?
- Does it explain how enquiries and complaints will be handled, set timeframes for responding, and identify possible remedies and complaint outcomes?
- Does it require that written reasons are provided in complaint resolution?
- What proportion of your requests for information is granted?
- Is the enquiry and complaint process adequately explained?
Is your agency collecting data about the costs attached to supplying information?
Can you provide any case studies concerning publication of PSI by your agency and the impact that it has had?
Benjamin, Solomon, R Bhuvaneswari, P Rajan and Manjunatha, Bhoomi: 'E-Governance', Or, An Anti-Politics Machine Necessary to Globalize Bangalore?, CASUM-m Working Paper, January 2007.
Clapton, Genevieve, Max Hammond and Nick Poole, PSI reuse in the cultural sector: Final report, Curtis + Cartwright report to the European Commission, May 2011.
Commercial Exploitation of Europe's Public Sector Information (Final Report), Pira International for the European Commission Directorate General for Information Society, October 2000.
Corbin, Chris, A review of indicators used in PSI studies (Version 4, Reference paper), European Commission Public Sector Information Group, November 2009.
Corbin, Chris, Public Sector Information: Economic Indicators and economic case study on charging models, (Version 2, Final Report), European Commission Public Sector Information Group, August 2010.
Dekkers, Makx, Femke Polman, Robbin te Velde and Mark de Vries, MEPSIR - Measuring European Public Sector Information Resources: Final report of study on exploitation of public sector information - benchmarking of EU framework Conditions, report for the European Commission, June 2006.
Ebrahim, Alnoor, and Rangan, V Kasturi, The limits of non-profit impact: A contingency framework for measuring social performance (Working Paper 10-099), Harvard Business School, May 2010.
eGovernment Economics Project (eGEP) -Measurement Framework Final Version, report by RSO SPA and Luiss Management for the eGovernment Unit, Directorate-General Information Society and Media, European Commission, May 2006.
Garcia Almirall, Pilar, Montse Moix Bergadà and Pau Queraltó Ros, The Socio-economic Impact of the Spatial Data Infrastructure of Catalonia, European Commission Joint Research Centre, Office for Official Publications of the European Communities, 2008.
Government 2.0 Taskforce, Engage: Getting on with Government 2.0, December 2009.
Houghton, John, Costs and Benefits of Data Provision: Report to the Australian National Data Service, Centre for Strategic Economic Studies, Victoria University, September 2011.
Lichtenstein, Jesse, 'Why open data alone is not enough', Wired, 28 June 2011.
The Power of Information: An independent review by Ed Mayo and Tom Steinberg, commissioned by the UK Cabinet Office, June 2007.
Newbery, David, Lionel Bently and Rufus Pollock, Models of Public Sector Information Provision via Trading Funds, report to the Department for Business, Enterprise and Regulatory Reform and HM Treasury, February 2008.
OECD Recommendation of the Council for Enhanced Access and More Effective Use of Public Sector Information [C(2008)36].
Office of the Australian Information Commissioner, Issues Paper 1 - Towards an Australian Government Information Policy, November 2011.
Office of the Australian Information Commissioner, Principles on open public sector information, report on review and development of principles, May 2011.
Office of Management and Budget, Open Government Directive, Memorandum for the Heads of Executive Departments and Agencies (M-10-06), Executive Office of the President, 8 December 2009
O'Hara¸ K, Transparent government, not transparent citizens: A Report on Privacy and Transparency for the Cabinet Office, September 2011.
Open Government and Public Value: Conceptualising a portfolio assessment tool, Center for Technology in Government, University at Albany, May 2011
Scheelong, Alexander and Philipp Girrger, Government 2.0 in beta phase: an analysis of eParticipation and web 2.0 applications of Germany's 50 largest cities and 16 federal states, Public sector study series, June 2010.
The commercial use of public information (Report OFT861), UK Office of Fair Trading, December 2006.
Thompson, Clive, 'How information can fuel jobs', Wired, 29 March 2011.
Wadhwa, Vivek, 'The coming death of open government', The Washington Post, 22 June 2011.
Weiss, Peter, Borders in Cyberspace: Conflicting Public Sector Information Policies and their Economic Impacts (Summary report) US Department of Commerce, National Oceanic and Atmospheric Administration, National Weather Service, February 2002.
Zappalà, Gianni, Solving social problems and demonstrating impact: A tale of two typologies (Briefing Paper 5) Centre for Social Impact, University of New South Wales, February 2011.
 Jesse Lichtenstein, Why open data alone is not enough, Wired, www.wired.com/magazine/2011/06/st_essay_datafireworks/ 28 June 2011.
 Government 2.0 taskforce, Engage: Getting on with Government 2.0, 2009, p 40 at www.finance.gov.au/publications/gov20taskforcereport/index.html.
Government response to the Recommendations of the Government 2.0 Taskforce at www.finance.gov.au/publications/govresponse20report/index.html#recommendation-06.
 Freedom of Information Act 1982, s. 3.
 Corbin, Chris, A review of indicators used in PSI studies (Version 4, Reference paper), European Commission Public Sector Information Group, November 2009, p 13.
 K O'Hara¸ Transparent government, not transparent citizens: A Report on Privacy and Transparency for the Cabinet Office, September 2011 at www.cabinetoffice.gov.uk/sites/default/files/resources/transparency-and-privacy-review-annex-a.pdf.
Open Government and Public Value: Conceptualising a portfolio assessment tool, Center for Technology in Government, University at Albany, May 2011, pp 6-7.
 Craig Thomler, blog comment, http://oaic.govspace.gov.au/2011/02/21/find-play-share/#comment-49 24 February 2011.
 Freedom of Information Act 1982, s 11B(4)(a).
 Solomon Benjamin, R Bhuvaneswari, P. Rajan, Manjunatha, Bhoomi: E-Governance, Or, An Anti-Politics Machine Necessary to Globalize Bangalore? CASUM-m Working Paper, January 2007 at http://casumm.files.wordpress.com/2008/09/bhoomi-e-governance.pdf.
 Dekkers, Makx, Femke Polman, Robbin te Velde and Mark de Vries, MEPSIR - Measuring European Public Sector Information Resources: Final report of study on exploitation of public sector information - benchmarking of EU framework Conditions, report for the European Commission, June 2006 p 15.
 These are One: open access to information - a default position, Five: discoverable and useable information and Six: clear reuse rights.
 OECD Recommendation of the Council for Enhanced Access and More Effective Use of Public Sector Information [C(2008)36] at p 4, at http://webnet.oecd.org/oecdacts/Instruments/ShowInstrumentView.aspx?InstrumentID=122&InstrumentPID=118&Lang=en&Book=False.
 Report on review and development of principles, pp 4–5.
 Examples include the publication by the Powerhouse Museum of some significant photographic collections under an open licence through Commons on Flikr (www.flickr.com/photos/powerhouse_museum/) and the presentation of excerpts from a selection of Australian films and television programs on australianscreen at http://aso.gov.au/.
 See Engage report, p 61.
 See Engage report, p 62.
 John Houghton, Costs and Benefits of Data Provision: Report to the Australian National Data Service, Centre for Strategic Economic Studies, Victoria University September 2011. This study is discussed further in section 4 below.
 See Engage report, p 46.
 Recommendations 6.12 and 6.13 of the Gov 2.0 Taskforce.
 See MEPSIR p 15.
 Gianni Zappalà, Solving social problems and demonstrating impact: A tale of two typologies (Briefing Paper 5) Centre for Social Impact, University of New South Wales, February 2011 at www.csi.edu.au/site/Knowledge_Centre/Asset.aspx?assetid=145e2b8d68c4a0b6.
 Zappalà, Solving social problems, p 3.
 The framework was first presented in Alnoor Ebrahim and V Kasturi Rangan, The limits of non-profit impact: A contingency framework for measuring social performance (Working Paper 10-099), Harvard Business School, May 2010 at http://hbswk.hbs.edu/item/6439.html.
 Zappalà, Solving social problems, p 7.
 Zappalà, Solving social problems, p6.
 Center for Technology in Government, Open Government and Public Value: Conceptualising a portfolio assessment tool, (Report), The Research Foundation of State University of New York, May 2011, p 3 at www.ctg.albany.edu/publications/online/pvat/.
Open Government and Public Value, p 6.
 See Open Government and Public Value, p 6.
 See eGovernment Economics Project (eGEP) –Measurement Framework Final Version, report by RSO SPA and Luiss Management for the eGovernment Unit, Directorate-General Information Society and Media, European Commission, May 2006, p 3 at www.google.com.au/url?sa=t&source=web&cd=2&ved=0CCkQFjAB&url=http%3A%2F%2Fwww.umic.pt%2Fimages%2Fstories%2Fpublicacoes200709%2FD.2.4_Measurement_Framework_final_version.pdf&rct=j&q=eGovernment%20Economics%20Project%20(eGEP)%2C%20Measurement%20Framework%20Final%20Version&ei=vNScToKXN4-ZmQXM54WNCQ&usg=AFQjCNGLBJPDlbiTG9Nj_ElD0LLLpaIk1A.
Measurement Framework, p 6.
Measurement Framework, p 24.
Measurement Framework, p 20.
Measurement Framework, p 24.
 Pilar Garcia Almirall, Montse Moix Bergadà and Pau Queraltó Ros, The Socio-economic Impact of the Spatial Data Infrastructure of Catalonia, European Commission Joint Research Centre, Office for Official Publications of the European Communities, 2008, p 11 at http://inspire.jrc.ec.europa.eu/reports/Study_reports/catalonia_impact_study_report.pdf.
The Socio-economic Impact of Spatial Data Infrastructure,p 30.
The Socio-economic Impact of Spatial Data Infrastructure, pp 36–37.
The Socio-economic Impact of Spatial Data Infrastructure,p 46.
 This figured is based on an estimated saving of 500 hours per month which, at a notional rate of €30 per hour, produces savings of over €2.6 million per annum. The Socio-economic Impact of Spatial Data Infrastructure, p 44.
 Open Government Directive, Memorandum for the Heads of Executive Departments and Agencies, Executive Office of the President, Office of Management and Budget (M-10-06), 8 December 2009, p 5. (www.whitehouse.gov/sites/default/files/omb/assets/memoranda_2010/m10-06.pdf).
 John Houghton, Costs and Benefits of Data Provision: Report to the Australian National Data Service, Centre for Strategic Economic Studies, Victoria University, September 2011, http://ands.org.au/resource/houghton-cost-benefit-study.pdf.
 Houghton, Costs and Benefits of Data Provision, p 8.
 Houghton, Costs and Benefits of Data Provision, p 11.
 Houghton, Costs and Benefits of Data Provision, pp 19–20.
 Houghton, Costs and Benefits of Data Provision, pp 31–32.
 Houghton, Costs and Benefits of Data Provision, p 39.
 Houghton, Costs and Benefits of Data Provision, p 44.
 Chris Corbin, A review of indicators used in PSI studies (Version 4, Reference paper), European Commission Public Sector Information Group, November 2009, http://old-proxy.epsiplatform.eu/content/download/34229/462486/version/1/file/PSI_Reuse_Indicator_Review_V4.pdf.
 These are the MICUS Study (2008), the Cambridge Study (2008), Akesdo study (2008), CUPI study (2006), MEPSIR study (2006), BGS study (2003), and the PIRA report (2000).
 Corbin, A review of indicators, p 13.
 Corbin, A review of indicators, p 13.
 Corbin, A review of indicators, p 10.
 Chris Corbin, Public Sector Information: Economic Indicators and economic case study on charging models, (Version 2, final report), European Commission Public Sector Information Group, August 2010, p 17, at http://old-proxy.epsiplatform.eu/content/download/64545/898326/version/1/file/EC_EconomicStudyReport_V2.pdf.
 See Corbin, Public Sector Information Economic Indicators, p 3.
 See Corbin, Public Sector Information Economic Indicators, p 16.
The commercial use of public information (Report OFT861), UK Office of Fair Trading, December 2006, at www.oft.gov.uk/OFTwork/publications/publication-categories/reports/consumer-protection/oft861.
Economic Value and Detriment Analysis (Annex G OFT861G), report prepared for the UK Office of Fair Trading by DotEcon, December 2006, at www.oft.gov.uk/OFTwork/publications/publication-categories/reports/consumer-protection/oft861.
 See Economic Value and Detriment Analysis, p 5.
 See Economic Value and Detriment Analysis, p 20.
 See Economic Value and Detriment Analysis, p 21.
 See Commercial use of public information, p 20.
 See Commercial use of public information, p 21.
 See Commercial use of public information, p 32.
 Makx Dekkers, Femke Polman, Robbin te Velde and Mark de Vries, MEPSIR – Measuring European Public Sector Information Resources: Final report of study on exploitation of public sector information – benchmarking of EU framework Conditions, report for the European Commission, June 2006, www.google.com.au/url?sa=t&source=web&cd=1&ved=0CBsQFjAA&url=http%3A%2F%2Fec.europa.eu%2Finformation_society%2Fpolicy%2Fpsi%2Fdocs%2Fpdfs%2Fmepsir%2Ffinal_report.pdf&ei=JdGcTsmiHevjmAWCxKCKCQ&usg=AFQjCNFMasJTLMzhbhIn-b8uqpCIUDddpA.
 Directive 2003/98/EC of the European Parliament and of the Council of 17 November 2003 on the re-use of public sector information is available at http://ec.europa.eu/information_society/policy/psi/actions_eu/policy_actions/index_en.htm
 See MEPSIR, p 11.
 These were business information, geographic information, legal information, meteorological information, social data and transport information (MEPSIR, p 14).
 See MEPSIR, p 15.
 The study identified the necessary economic conditions as Actual demand: Equal and fair access to information will boost the actual demand and Economic results: This will eventually translate into direct (more turnover for re-users) and indirect (more commercial activity based on public sector information) economic results.
 See MEPSIR, p 16.
 See MEPSIR, p 33.
 See MEPSIR, p 33.
 See MEPSIR, p 33.
 See MEPSIR, p 35.
 See MEPSIR, p 45.
Commercial Exploitation of Europe's Public Sector Information (Final Report), Pira International for the European Commission Directorate General for Information Society, October 2000, at ftp://ftp.cordis.europa.eu/pub/econtent/docs/commercial_final_report.pdf.
 See Commercial Exploitation of Europe's Public Sector Information, p 8.
 For instance, Pira International notes (Commercial Exploitation of Europe's Public Sector Information, pp 23–24) that 'according to the Ordnance Survey report, the annual value of its maps to the United Kingdom economy is between €130bn and €211bn – or between 10% and 15% of annual GDP. Given the large number of other resources which are vital to the workings of the modern economy, these numbers are obviously far too high. Repeating the exercise for all PSI domains would for instance, lead to value figures 2–3 times greater than national income.'
Commercial Exploitation of Europe's Public Sector Information, p 109.
 Peter Weiss, Borders in Cyberspace: Conflicting Public Sector Information Policies and their Economic Impacts (Summary report), US Department of Commerce, National Oceanic and Atmospheric Administration, National Weather Service, February 2002 at www.weather.gov/sp/Borders_report.pdf.
 David Newbery, Lionel Bently and Rufus Pollock, Models of Public Sector Information Provision via Trading Funds, report to the Department for Business, Enterprise and Regulatory Reform and HM Treasury, February 2008, at www.berr.gov.uk/files/file45136.pdf.
 Newbery, Bently and Pollock, Models of Public Sector Information Provision, pp 1, 19–20.
 See Weiss, Borders in Cyberspace, 2.
 See Weiss, Borders in Cyberspace, pp 9–12.
 See Weiss, Borders in Cyberspace, p 13.
 Weiss (Borders in Cyberspace, pp 13–14) cites the case of two counties in Wisconsin, one of which took a cost-recovery approach to its digitised aerial photographs while the other charged only marginal costs. In the cost-recovery case, the photographs were used by a small group of specialists; when the photographs were available at marginal cost, a much wider range of individuals used them for many new projects.
 See Weiss, Borders in Cyberspace, p 17.
 Newbery, Bently and Pollock, Models of Public Sector Information Provision, pp 1, 19–20.
 'Total welfare' is described as 'an overall measure which incorporates all of the other changes into a single value, usually presented in monetary terms for convenience of comprehension'. Newbery, Bently and Pollock, Models of Public Sector Information Provision, p 15.
 OAIC, Principles on open public sector information, May 2011, pp 26–27.
Freedom of Information Act1982 s 3.
 For the purposes of this question 'information holdings' are principally reports, publications, datasets, maps and images.