[cs_content][cs_section parallax=”false” style=”margin: 0px;padding: 0px 0px 45px;”][cs_row inner_container=”true” marginless_columns=”false” style=”margin: 0px auto;padding: 0px;”][cs_column fade=”false” fade_animation=”in” fade_animation_offset=”45px” fade_duration=”750″ type=”1/1″ style=”padding: 0px;”][x_custom_headline level=”h1″ looks_like=”h1″ accent=”false” class=”cs-ta-center mvs”]On Working with Open Data[/x_custom_headline][cs_text]from ’13 to ’16 I’ve been working on Portuguese Open Data initiatives, namely acting a Project Lead for the Public Open Data Portal, and representing the country on a series of internacional bodies and projects. This text is about what open data is, my experiences in the field and some personal considerations on the subject. All opinions are my own.
[dropcase]A[/dropcase]round mid-2013, after, after roughly seven years working in Marketing & Communications, I was offered the opportunity to change areas inside my organization (AMA).
It wasn’t a change I was actively pursuing, but one that made sense at the time. As a public body, we were under peak-Troika austerity measures, and the Communications budget – as is bound to happen – was a prime target for cuts. There wasn’t a lot (of new stuff) to do. So, even if was a bit weary of leaving my home turf, I still jumped at the opportunity of joining the Innovation and International Relations team.
I was assigned to the development of Open Government projects and initiatives, among other things. In practice that meant taking over the project management of the Portuguese Public Open Data portal – dados.gov (launched two year earlier) and work on a broader strategy for championing for transparency and accountability of the Public Administration.
Open Government is a burgeoning field of Public Administration management. It is related to perennial democratic concepts such as Freedom of Information, accountability and transparency of the state, but for the last decade or so it also became closely tied with digital tools and ICT policies.
Open Data
Open Data is one of the most visible fields of Open Government initiatives. At its inception, Open Data is any data that you are free to use without any sort of constrains. It’s data you can use to build apps and make money out of them (if you want). To work on academic projects. Or just lovingly gaze at in the privacy of your home. There are no strings attached to use this data. You don’t need to pay, you don’t need to register, you just grab it. In the Public Administration context, we usually talk about Public Open Data (I will be using both terms interchangeably).
Public Open Data is data that is produced or results of government work. Think about the data that is collected in census. Or the data regarding the physical locations and addresses of government buildings and infra-structure. Public transportation timetables. Official building energy consumption figures. Information about business registrations. And so on. Think of all the data that hundreds of public institutions have to collect, work with and publish. That’s a lot of data. All of this data is part of what, in Europe, we call Public Sector Information (PSI). PSI includes all of this data, but also historical documents, films, and stuff like that. And do you know who owns this data? No, not government. Governments should be the custodians of public data, but the owners are all of us – the taxpayers.
The reason why Public Open Data is the sexy fledgling of Public Sector Information is that Open Data, at its best format, is meant to be machine-readable and therefore easily streamlined to build those before mentioned apps, websites and platforms.
There is an interesting (or sterile) debate about what really constitutes machine-readableness. It its most common definition, machine readable information is data that can be extracted and processed by a computer program (usually formats like XML or JSON). Some people argue that a scanned pdf is not really a machine-readable format, while other point that pdfs are opened by computers, which are machines (thus…). I don’t really feel qualified to participate on that debate, but you know what’s even more terrible? When data is not available at all, or just buried inside some institutional website or closed room.
and speaking of burying data, that’s when we get to
Open Data Portals
Public Open data portals is where people, usually developers (i.e. geeks, coders, or anyone who know their way around a ‘webservice’), come and get open data. Portals are importante because the organize, catalog and aggregate information. That’s what we have been working on at dados.gov for the past years. Dados.gov is the central public open data portal in Portugal. It’s challenging work, specially when you are doing it with minimal resources in an emerging (and fast-changing) field. The portal was launched in 2011 and took off with around 100 datasets. Five years later we are approaching the 800 dataset mark from over 15 different data suppliers across multiple ministries and organizations.
Who benefits?
There are usually two ways of looking at the value of Open Data. One is the obvious value of transparency and accountability. The other is the economic potential of opening up data free of reuse restrictions (i.e. for commercial purposes). Of course they are not mutually exclusive, but at one point the latter became one the main flags of the movement. It’s a great carrot for gathering political support. The problem? It takes time to happen. So around 2013, 2014, cynics started questioning the value of open data. Where were all the trillions promised by the studies on open data economic value? The string of exciting and rich startups built on open data?
It’s good to have cynics. I am known to be a particular practitioner of self-questioning and skepticism. But I don’t really think that questioning the potential economic value of open data is questioning the value of open data. Besides having an intrinsic value of transparency and accountability, Open Data can also be a driver of innovation and efficiency for the Public Administration itself. This is something that we have found to be very relevant in the Portuguese context, and a strategy that we actively pursued for the past years.
While responsible for the general promotion and the development of Open Data in Portugal as part of its responsibilities in the fields of public administration ICT policies and electronic government, AMA has been promoting the open data portal as a public shared service for all of the Portuguese Public Administration. Building on the importance of sharing information and promoting its reuse, public organizations are encouraged to use dados.gov as a means of making data available on a robust platform that turns tables into webservices, can be accessed via an API and is covered by a Creative Commons BY license.
Countries have been following different strategies in accordance to their political and organizational context. In some nations politicians raised open data as a political flag and encouraged (or ordered) ministries and departments into opening data. Some of these countries also moved quickly into putting legislation into place that enforces open data policies or reuse of information standards. Federal or regional countries often move at different speeds, but are also encouraged by an internal competition spirit. From 2011 to 2016, in Portugal we have been following mostly a bottom-up approach, working open data at the Public Administration level.
Engaging different people
While frustrating at times, this has been an amazing experience, forcing us to engage the public administration at different levels and promoting something that (we believe) is paramount to the greater good and public service. The work as been a mixture of project management, product development, content management, technological evangelism and, at times, cold call selling. There is a portal and a myriad of webservices to maintain and keep working, and a development pipeline to look for in the next years. There are workshops, conferences and one to one meetings where we try to bring everyone on board. And there are a lot of blank stares, “I’ll call you later” and “what’s in this for mes” reactions that we get from would be data suppliers.
Working with little resources had made us adapt and try to make the most with less. We used a what we called “Government as a Developer” strategy, where we identified high profile web / app development projects within government and then nudged open data as a building block of such projects. We did that with the Citizen Map app and the Local Transparency Portal, opening up most of the data that powers those applications. This means that anyone can access and use that same data to build similar (or better) stuff.
Engaging stakeholders is another classic tenet of open data initiatives. In Portugal we have been focusing mainly on the Public Administration and Civil Society sectors. Besides all the work we’ve been doing at government level, we also have made some occasional efforts in reaching out to hacktivist organizations, data journalists, transparency think tanks, and the like. The academic and business sectors also figure on our priority list, but they require a more complex approach and some investment – so we’ve been focusing on the low-hanging fruit.
In the past two years we made significant efforts to reach out and listen to the community. We met with most of the people mentioned above and, most importantly, sat them together and discussed the future vision for open data in Portugal. We followed design thinking methodologies and brainstormed the crap out of the past, present and future of opening up data in Portugal. This allowed us to further access the needs from our partners on the Public Administration, which included the need for training and technical support in developing webservices, and also the general feeling from civil society that data portals need political and legal enforcing.
On a personal level, it’s an incredible pleasure to attend community events and meetings about open data and hear from the people that have diverse interests in this subject. I’ve talked with hacktivists, students, designers, researchers, entrepreneurs, and they all had the same positive attitude in common – even if they were asking the hard questions. And of course we appreciate it if people are asking questions, it means they are interested and we can build momentum about this.
and thinking at the global level
It’s not easy when you are working on a project so ambitious and that can – at most – only occupy 1/4 of your time. Of course there must be priorities and, contrary to popular belief, there is a lot of work to be done on the Public Administration in the digital and modernization fields. So we have been handling matters with a small team of people and resource pool, doing our best (or so I believe) with the little we have. Fortunately, Open Data is an exciting new field of government and has a lot of recognition from international organizations. As such, there are several working groups and projects at the European and wider level, and we have been participating in some of them.
There’s SHARE-PSI 2.0, a Thematic Network sponsored by the European Commission and gathering 47 organizations (public bodies, companies, universities, among others) from 26 different countries, where I represented AMA and the sole Portuguese participation (and where we lead one of the Work Packages of the project). There’s also the OECD Expert Group on Open Data, or the European Commission PSI Working Group, where I have been invited as a national representative. Having the responsibility to represent my organization and my country on these different bodies has been an amazing and very enriching experience. Most importantly, it’s an opportunity to breathe, eat and sleep open data for two, sometimes three, days straight.
Speaking to, sharing experiences with, and learning from: experts, politicians, PhDs, data specialists, bureaucrats and mostly nice and enthusiastic people about open data, from all over the world, has been the best open course on this matters that I could wish for. Most recently, in April this year, I was also invited to speak as an Expert on Open Data at an OECD organized conference in Tunis, which further expanded my convictions on the importance of open data in democracy development and building a better government.
Now what for Open Data?
From being a pioneer country in 2011, Portugal has inevitably lost a bit of momentum in the last years in the context of intelligent spending by a lot of countries. This is not necessarily bad, because as an emerging area, criteria is fast somehow changing and sprinting is still rewarded. As such, there are a lot of mistakes to draw from and a lot of insight to be gained. Jumping to the top of the rankings isn’t that hard, but ensuring that you have a long-term strategy in place is the real challenge.
I believe that we are at the nexus of a new kicking point for a nation wide Open Data strategy. The PSI Directive will soon be transposed to Portuguese law, the startup scene in Portugal is entering a new maturity stage, and the Public Administration is now showing signs of interest in embracing these principles (both centrally and locally). Moreover, there seems to be an understanding that you can’t Open Data without addressing other fundamental issues regarding ICT management (i.e. interoperability) in government systems.
I’ve been purposely avoiding the “cultural challenge” of open data, since it could warrant another 2,000 words. Yes, it’s one of the major obstacles, specially with an ageing and insulated Public Administration, but one that can be tackled with policy and management. Public servants aren’t the issue here, but the way work is organized. And that can be changed.
While still at an early stage, I think Open Data represents the first step in advancing towards an openness culture in the Public Administration. There are many benefits to going open, such as transparency, economy, but also efficiency. Hopefully we will soon start thinking about Government as a Platform, about opening up APIs for public services, sharing the source code for major government platforms. Maybe government will also start employing some of the brilliant minds that graduate from Portuguese university’s excelente computer engineering programs. It may take a while, but I’m betting that we will all be here to see it, and contribute to it.[/cs_text][/cs_column][/cs_row][cs_row inner_container=”true” marginless_columns=”false” style=”margin: 0px auto;padding: 0px;”][cs_column fade=”false” fade_animation=”in” fade_animation_offset=”45px” fade_duration=”750″ type=”1/1″ style=”padding: 20px 0px 0px;”][cs_text class=”cs-ta-center”][icon type=”arrow-left” style=”font-size: 40px; “]
BACK TO TEXTS[/cs_text][/cs_column][/cs_row][/cs_section][/cs_content]