Technology’s Role in Creating a Better Government - 3 videos

3 videos from the civic tech space:

Google is a big supporter of Code for America and we've invited their co-founder Jennifer Pahlka to do an interview with Stephanie Hannon for the Women TechMaker program.

"Coder and activist Jennifer Pahlka believes that apps, built quickly and cheaply, are a powerful new way to connect citizens to their governments. Google's Steph Hannon chats about the future of Code for America and how to get involved."

We've also invited her to do a Techtalk at Google campus, and this is the video.

"Jennifer Pahlka is the founder and executive director of Code for America. She recently served as the US Deputy Chief Technology Officer in the White House Office of Science and Technology Policy. She is known for her TED talk, Coding a Better Government, and the recipient of several awards, including MIT’s Kevin Lynch Award, the Oxford Internet Institute’s Internet and Society Award, and the National Democratic Institute’s Democracy Award. She spent eight years at CMP Media, where she ran the Game Developers Conference, Game Developer magazine, Gamasutra.com, and the Independent Games Festival. Previously, she ran the Web 2.0 and Gov 2.0 events for TechWeb, in conjunction with O’Reilly Media. She is a graduate of Yale University and lives in Oakland, Calif. with her daughter, her fiance, and seven chickens. 

Can government be run like the Internet, permissionless and open? Coder and activist Jennifer Pahlka believes it can — and that apps, built quickly and cheaply, are a powerful new way to connect citizens to their governments — and their neighbors."

Julia Kloiber works at Open Knowledge Foundation and is the project lead for Code for Germany, which she describes in her recent TED talk "Let's build better digital tools for our cities!"

"When it came to getting public consensus on a complex plan to convert the epic Tempelhof Airport into something new, Berlin sought an effective way to communicate the nuances of the various proposals. The solution? A computer coder synthesized the information from all the options into a simple infographic, leading to an informed vote by the public. Julia Kloiber argues for the release of government data so coders can unscramble it for the public good - in Berlin and beyond."

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

A necessary rebellion

Talk by Joi at Poptech, "A necessary rebellion"

Some quotes from the talk:

"Question authority and think for yourself: You don't get Nobel prize for doing what you are told- you get Nobel prize for questioning authority and thinking for yourself. But somehow, we made this education system that requires you to have the right answer, to be obedient and to do things the way everybody else does them because it's the way you're supposed to do it."

"Fragile is a something that breaks under stress, and not robust against chaos. But robustness and resilience is not the opposite of fragile, because robustness and resiliency survives chaos. "Antifragile" are things that gets stronger when there's stress or are attacked. For example immune system gets smarter and stronger when they get attacked. Network security is another one. Think about "Antifragile" applied to all the systems including governments, education... the world is complex, stressful, and if you try to eliminate all rebellion and all the stress, you come up with a relatively fragile system."

"Learning over education: Education is something people do to you, learning is what you do to yourself. We need creative learning: In the old days, mass production, post industrial, pre-robot, pre-AI ages, you needed people who are obedient, could do repetitive task, weren't rebellious. But now, people should be more creative, because creative things are what the computers can't do. All the repetitive jobs are going to be taken over by computers."

"4Ps of Creative Learning = Projects, Peers, Passion and Play. Learning out of context doesn't work- learning through doing something you can learn, hence projects are important. Peers: teaching each other is a great way to learn. Passion is important. Play: if you put pressure on somebody and give financial rewards, they will do simple tasks more efficiently but it will take longer to do creative tasks. We're used to pressure people to be on time and be obedient that we have stamped out creativity on the kids. Unless we can transform our children, education system and workforce to a more rebellious and robust system, we will lose our jobs to computers and robots, and it's the only way we can survive."

Japan is a country that grew lots of people who are obedient, can't say no, and aren't allowed to say no in schools and workplaces. "Too tall a pile is hit on the top."

On the other hand, Japan is the country that is full of high quality amateur creativity in the fields of music, art, illustration and cartoons, animations, novels, software, hardware and many more fields in art. The amount of creativity that you see in places like Comic Market and Maker Faire Tokyo are just amazing. Of course many commercial products are exported out of the country, but the layers and layers of amateur creativity flourishing in places that are not mass production. I feel a lot of potential and hope for those creativity to lead to next generation industries.... from the awesome Japanese "rebellions" :)

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

The Internet's Own Boy: The Story of Aaron Swartz

I've supported Kickstarter for this project and saw it when it was released, but I realized they now have subtitles in English, Spanish, Russian, French, Italian, Turkish, Chinese, Japanese, and Vietnamese. More reason for everyone in the world to watch this video.... and think about what you can do, what we can do, to make the world a better place. The world lost Aaron. We have to continue what he wanted to make happen.

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

How Open and Inclusive Can Government Procurement Be? - Example from Fukushima

There are lots of projects to attempt transforming procurement.

Identifying the problem, and example from Philadelphia

My favorite is this project by Mark Headd former Chief Data Officer of Philadelphia where they did a GitHub-based procurement - "Redesigning Government".



Let's look at the problem: that the government procurement is broken, "fear-based procurement" is taking place, meaning people working in the government in general doesn't want to take risks so they stick to less risky procurement process, with old-style vendors they know, and the newer innovative startups has no way to contribute.

In the case of Philadelphia, they used GitHub for procurement for the following reasons:
1) They wanted to use a platform that would resonate with technologists, and wanted to work with firms that believed in what they believed- "we value open source software, we value collaborative software development, we value awesome things, and existing procurement processes don't allow us to find firms that share our values" says Mike.
2) They wanted to have some insight into the quality of other work done by bidders (to be able to look at their public repos to see what kind of solutions they were working on and how active they were in participating in other projects)
3) They wanted to see creative responses - something that potential vendors probably had not seen from a city government before.

The project they decided to use this process was called myPhillyRising, and they posted their RFP (what they want to build) on GitHub Gist.

https://gist.github.com/PhillyCDO/4666456

Anybody can see it, anybody can ask question or clarification can post public comment on this Gist so that everyone will see those questions and everyone will see those answers. If vendors wanted to bid, they had to create a GitHub repository, and the proposals should be in that repository. Mark and the team used GitHub issues to evaluate the vendor proposals, tagged them and issued bugs to assign the evaluation processes to the team.

Some firms just put a pdf document on their repository, some of them submitted actual code as submission, and the city can install and test them. Another vendor submitted a website that had user stories. Changing the "deign of the procurement process" itself allowed the city to reach those creative firms with shared values.

I think this is fascinating. You can read more about it here:
Experiments in GitHub Based Procurement

Many examples around the world

But this is not the only way to change government procurement - there are many different approaches taken by different governments.

How Barcelona and Philadelphia Are Turning Procurement Upside Down

It talks about how the cities opened their RFPs to everyone to be inclusive to the local citizens, and not specifying the solutions they are looking for but specifying the problems they want to solve made bidders more creative: “If we think about procurement as less about buying solutions and more about solving problems, I think we can open ourselves up to a whole variety of innovations”

And they introduce at tweet of a woman reading the RFP over breakfast: “Suddenly, procurement has gone from something completely horrible to something people can imagine participating in.”

Example from Namie, Fukushima

I wrote about Namie in the past on this blog post: it's a town very near Fukushima nuclear power plant, over 37% of the population is over 60 years old, all of the citizens are in evacuation in other places due to the radiation, and the town is completely in a diaspora. Namie is a small town with around 10,000 households/ 20,000+ citizens, and the town decided to distribute each household a tablet to communicate among each other as well as the town to distribute information. The project has 290,000,000 yen (approx. 2,900,000 USD) as the budget, from the Recovery Agency.

With such aging society, inclusion is extremely important. Here is how they are running the procurement process, together with Code for Namie team.

They have this website that explains to the citizens what this project is about, and the process of procurement.

http://www.town.namie.fukushima.jp/soshiki/2/201405tablet.html

1. They ran 6 ideathons where citizens, developers and designers gathered to discuss ideas for what they want the tablet to be doing in order for it to be useful for them.
2. They summarized the ideas to select the theme for hackathon.
3. They ran 2 sets of 2-day hackathons to develop prototype apps, and had the citizens evaluate those apps.
4. Based on those evaluations by the citizens, the town created RFP and invited vendors to submit proposals for creating applications and running the distribution and operation of the project.
5. All of the households receive application form to apply for the tablet, and monitor households will be able to start using the test device. Events to experience the tablet will be held.
6. Based on the monitor and events, adjustments will be made, and the tablets will be distributed.

Photo from one of the ideathons:

All of the RFP and Q&A results can be seen here:

http://www.town.namie.fukushima.jp/soshiki/1/20140801-proposal.html
http://www.town.namie.fukushima.jp/soshiki/1/8133.html

This is the actual specification from RFP.

http://www.town.namie.fukushima.jp/uploaded/attachment/2870.pdf

Some of the functions includes:
1. Local news distribution
2. Radiation level information distribution
3. Local government information distribution
4. Inter-household SNS
5. Function to increase usage (support function to support non-technical people using characters)
6. Slideshow
7. Inter-citizens SNS (optional)
8. Namie archive (optional function to preserve photos, videos and cultural assets from Namie)

These are the functions that actually came from the citizens' demand. You can see the "idea sketch" from ideathon incorporated in the RFP.

They also attached the "persona" analysis of the users, for the vendors to envision the user scenario.

http://www.town.namie.fukushima.jp/uploaded/attachment/2873.pdf

1. Community leaders, old couple living in temporary housing in Fukushima City
2. Community creators, young mom & daughter in Chiba, with high IT literacy
3. Old man living alone in Nihonmatsu, enjoying single life
4. Family living in Sendai, adjusting their lifestyle based on the children
5. "SOS" type- evacuated to Saitama, but cannot adjust to the locals and having trouble with communication

There were 6 companies that made proposals, all of the proposals can be seen here: the presentation files and video of their actual presentation to the town. You can see all of the scores, overall evaluations and detailed evaluations here. It is extremely transparent.

http://www.town.namie.fukushima.jp/soshiki/1/8028.html

Watching the video archive, you realize that at the beginning of the presentation, the vendors are told "citizens from Namie are in the room listening, so please do not use technical terms- please make your presentations understandable to everyone." One of the vendors started using lots of jargons in their Q&A and one of the person from the audience replied "I did not understand a word you said. Can you explain in plain sentences?"

Also, there is one person in the room that keeps asking the same question to the vendors: "Do you know the percentage of the population of Namie over 60 years old?" and "Do you know what percentage of the citizens have evacuated outside Fukushima?" These information were all in the first couple of pages in the RFP, but surprisingly some of them could not answer.

This is the slides and video of Fujitsu that won the bid.

Slides:
http://www.town.namie.fukushima.jp/uploaded/life/8028_24220_misc.pdf
Video:
https://www.youtube.com/watch?v=AL8Ds_tNtk0

This is the short summary evaluation doc of all of the vendors.

http://www.town.namie.fukushima.jp/uploaded/life/8028_24299_misc.pdf

This one is the detailed one.

http://www.town.namie.fukushima.jp/uploaded/life/8028_24201_misc.pdf

So the first question would be "Why the heck is NTT Docomo's score so low??" NTT Docomo is the largest mobile carrier in Japan, and clearly should have done lots of app development and system integration projects like this. They actually have a lot of experience- in fact, they already have ASP service that has many of the functions needed in Namie, and have already implemented in other town in Fukushima, that they can leverage. Their proposal was basically using this ASP service, customizing to Namie. At first thought, this makes sense, and their presentation seems legit. Instead of reinventing the wheel, let's use existing apps and customize it. What they did NOT realize is that this project was supposed to be agile, collaboration project involving the citizens to prototype, iterate, test, and give the feedback loops for further iteration, although it was written clearly in the RFP. It was not meant to be "giving" the system, but "building together". Also, the citizens did research about how Docomo's similar system in other town of Fukushima was doing, and it was not functioning well. Despite using the existing system, their initial cost and operational cost was expensive.

To summarize, almost all of the steps of the procurement is open, and citizens are inclusive of that process through the following steps:
1. Citizens were involved in shaping the RFP by giving ideas of "what they need".
2. Citizens were involved in hackathons to give feedbacks to the prototypes created.
3. Citizens were involved in evaluation of the vendors by joining and asking questions at their  presentation.
4. Citizens will be involved in using the app early as monitors, join "experiencing events" to give feedbacks for iteration before the final rollout.

Sure, they didn't use GitHub, all of their docs are in pdf format, but I think it is subtle in this case: this is about a small and aging town, with all of their citizens in diaspora, almost nobody is tech-savvy and they have little experience developing systems or apps, but they are working together with the citizens to build something useful for their communication and knowledge sharing which is essential to their future.

Getting the government procurement right, together

Hopefully, those various examples of "changes to government procurement process" will help make more success cases to government procurement projects- according to a research, "94% of large federal information technology projects over the past 10 years were unsuccessful — more than half were delayed, over budget, or didn’t meet user expectations, and 41.4% failed completely".

More on that here: Why the Government Never Gets Tech Right

Instead of complaining about the failure rate, we have to work together to fix it.

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

Lawrence Lessig Interviews Edward Snowden

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

Silicon Chef - Women's hardware hackathon

I joined Silicon Chef- women's hardware hackathon this weekend.

Our team "Nature's Lullaby" made a musical instrument using Adruino, Max, light sensor, flex sensor and leap motion. 

1. Getting Arduino setup:

Install Arduino
http://arduino.cc/en/main/software

FTDI driver
http://www.ftdichip.com/Drivers/VCP.htm

SparkFun Inventor’s Kit Guide
http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Kits/SFE-SIK-RedBoard-Guide-Version3.0-Online.pdf

2. Making the LED blink, and change the parameters based on photo resistor and flex sensor

Hello world- making the LED blink:

void setup()
{
  pinMode(13, OUTPUT);
}

void loop()
{
  digitalWrite(13, HIGH);   // Turn on the LED
  delay(100);              // Wait for one second
  digitalWrite(13, LOW);    // Turn off the LED
  delay(100);              // Wait for one second
}

Using photo resistor to change the light level of LED:

const int sensorPin = 0;
const int ledPin = 9;

int lightLevel, high = 0, low = 1023;

void setup()
{
  pinMode(ledPin, OUTPUT);
}

void loop()
{
  lightLevel = analogRead(sensorPin);
  manualTune();  // manually change the range from light to dark
  analogWrite(ledPin, lightLevel);
}

void manualTune()
{
  lightLevel = map(lightLevel, 0, 1023, 0, 255);
  lightLevel = constrain(lightLevel, 0, 255);
}

void autoTune()
{
  if (lightLevel < low)
  {
    low = lightLevel;
  }

  if (lightLevel > high)
  {
    high = lightLevel;
  }
 
  lightLevel = map(lightLevel, low+30, high-30, 0, 255);
  lightLevel = constrain(lightLevel, 0, 255);
  }

3. Hooking up Arduino with Max:

Install Max
http://cycling74.com/downloads/

Fetching data of Arduino pins (in this case, data of photo resister) to Max

Maxuino 
http://playground.arduino.cc/Interfacing/MaxMSP

 /*
 *  Arduino2Max
 *  Copyleft: use as you like
 *  by Daniel Jolliffe
 *  Based on a sketch and patch by Thomas Ouellet Fredericks  tof.danslchamp.org
 */

int x = 0;                              // a place to hold pin values
int ledpin = 13;

void setup()
{
  Serial.begin(115200);               // 115200 is the default Arduino Bluetooth speed
  digitalWrite(13,HIGH);              ///startup blink
  delay(600);
  digitalWrite(13,LOW);
  pinMode(13,INPUT);
}

void loop()
{
if (Serial.available() > 0){         // Check serial buffer for characters      
    if (Serial.read() == 'r') {       // If an 'r' is received then read the pins  
for (int pin= 0; pin<=5; pin++){      // Read and send analog pins 0-5
    x = analogRead(pin);
    sendValue (x);
    }
for (int pin= 2; pin<=13; pin++){     // Read and send digital pins 2-13
    x = digitalRead(pin);
    sendValue (x);
    }
    Serial.println();                 // Send a carriage returnt to mark end of pin data.
    delay (5);                        // add a delay to prevent crashing/overloading of the serial port
  }
 }
}

void sendValue (int x){              // function to send the pin value followed by a "space".
 Serial.print(x);
 Serial.write(32);
}

Added a patch to change photo resister data to audio
(practically this is a theremin)

Fixing the patch to get various output sounds based on the light parameter and flex parameter.

4. Configure Max to read sample audios and remix

5. Setting up Leap Motion

https://developer.leapmotion.com/

Now we can do gesture input.

6. Making Leap Motion talk with Max

https://github.com/derekrazo/o.io.leap
http://cnmat.berkeley.edu/downloads

Adding Leap input to change the pitch of the sound, configured in Max.

Result:

We've got 3 levels of thresholds for light sensor, with frog sound and nature sound (as a lullaby to put you into sleep...), with additional strong light from flashlight app of Android we'll get a cat's meow sound (to wake you up), with flex sensor we'll get a rooster sound (if you were not waken up by the cat) and with Leap motion we can make that rooster sound changed to a digital squeak (to wake up the zombies).

[photo by Hackbright Academy]

It was not my first time playing with Arduino and Leap Motion, but my first time using Max. We had 2 experts on our team so they were able to figure out many things on the fly- I think I should get back to the elementary level and try out using it a bit more. Many thanks to the team for helping me learn!

[photo by Hackbright Academy]

Photos from the event:

Women building things!












Demos!













Note from TechTalk:

There are lots of cool events happening all around the world!

Nodebot is a global community of developers working on robots powered by JavaScript.

Nodecopter is an event with developers making something with AR Drone & Javascript



Noderocket- Javascript powered missiles



Robotsconf



JS Conf robot soccer game



Electric Imp

https://electricimp.com/

We set it up and ended up not using it this time, but it enables you to connect to wifi... will use it somehow for other projects :)

Max Shortcuts

Useful shortcut info,  thanks to Erin!

n = new object
i = integer object
f = floating point number object
m = message (for sending messages like t (for toggle) or b (for bang) or $1 (for use first incoming piece of data) etc.)
b = button
t = toggle
c = comment box (text only, for commenting)
CMD + E = toggle lock/ unlock patch
CMD + OPT + i = open inspector (use with an object highlighted--will open the inspector window for individual settings for that object)
CMD + SHIFT + h = open help (use w/ an object highlighted--will open help window for selected object)
CMD + SHIFT + r = open reference (same as above, but for reference)

My post from last year's Silicon Chef

http://fumiopen.blogspot.com/2013/10/silicon-chef-hardware-hackathon.html

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

Code for America SF brigade hacknight memo

From my notes at meetup in San Francisco tonight: Jen Pahlka and Tim O'reilly were the speakers.



- Let's put technology to the rightful place. It's not just about making money -technology can make society better.

- Should we be building something or reusing? Should governments be using taxpayers' money to build custom-made apps for each city?

- Change in vendor ecosystem is needed. 21st century governments can't survive without digital skill sets, tinkerers and builders.

- Government was our original means of collective action. It was built to do things that none of us could do individually. It’s wonderful to work together.

- Q: Technology will take away government people's jobs? A: Redeploy people to focus on outcome and actual people, don't get rid of them.

- Looking back on how Open Government advocacy started- it started by showing off people who are doing great things in the government, celebrate, and tell stories. Then, others who saw it will start doing it. That's how movements starts. That's how Tim started Open Source movement and other movements.

- 18F is deployment and USDS (US Digital Service) is about strategy and oversight. UK's GDS (Government Digital Service) actually does both.  Government realized they must spend less time just talking and more getting shit done ;)


 Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

Guides on "How to run Civic Hack Nights"

There are many guides on "how to run civic hack nights" recently, so I decided to compile a list here:

"How to get started in civic hacking" by Andrew Hyder

How to: Hack Night

• Check out Code for America’s Events Guide to help you with logistics such as finding a venue, promotion, and more.
• Watch Code for America’s Brigade Hangout on “How-To Hack Night” and read the transcript.
• Watch the GitHub 101 Video — Occasionally, Chicago will have a class on a certain aspect of civic hacking. This video is a recording of Derek Eder’s GitHub 101 class.
• Read A Year of Chicago OpenGov Hack Night — Blog post about the one year anniversary of Chicago’s OpenGov Hack Night, including lessons learned.
• Read How-To: #BetaNYC hacknight — Document that outlines how betaNYC organizes its weekly hack night.
• Check out a Code for America Brigade event close to you and ask its organizers about their best practices.

How To: Welcome New Members

Civic Hacking 101 by Christopher Whitaker

How to Hack Night panel

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

Guides to Publishing Open Data

As you can see from US City Open Data Census, many cities are making their data open.
http://us-city.census.okfn.org/

There are many guides and best practices learned from those cities, to publishing open data that countries/states/cities. Summary and links below for those interested!

Open Data Policy Guidelines by Sunlight Foundation
http://sunlightfoundation.com/opendataguidelines/
(CC-BY Sunlight Foundation)

What Data Should Be Public

1. Proactively Release Government Information Online
2. Reference And Build On Existing Public Accountability And Access Policies
3. Build On The Values, Goals And Mission Of The Community And Government
4. Create A Public, Comprehensive List Of All Information Holdings
5. Specify Methods Of Determining The Prioritization Of Data Release
6. Stipulate That Provisions Apply To Contractors Or Quasi-Governmental Agencies
7. Appropriately Safeguard Sensitive Information

How to Make Data Public

1. Mandate Data Formats For Maximal Technical Access
2. Provide Comprehensive And Appropriate Formats For Varied Uses
3. Remove Restrictions For Accessing Information
4. Mandate Data Be Explicitly License-Free
5. Charge Data-Creating Agencies With Recommending An Appropriate Citation Form
6. Require Publishing Metadata
7. Require Publishing Data Creation Processes
8. Mandate The Use Of Unique Identifiers
9. Require Code Sharing Or Publishing Open Source
10. Require Digitization And Distribution Of Archival Materials
11. Create A Central Location Devoted To Data Publication And Policies
12. Publish Bulk Data
13. Create Public APIs For Accessing Information
14. Optimize Methods Of Data Collection
15. Mandate Ongoing Data Publication And Updates
16. Create Permanent, Lasting Access To Data

How to Implement Policy

1. Create Or Appoint Oversight Authority
2. Create Guidance Or Other Binding Regulations For Implementation
3. Incorporate Public Perspectives Into Policy Implementation
4. Set Appropriately Ambitious Timelines For Implementation
5. Create Processes To Ensure Data Quality
6. Ensure Sufficient Funding For Implementation
7. Create Or Explore Potential Partnerships
8. Mandate Future Review For Potential Changes To This Policy

Open Data Playbook by Code for America- Open by Default [beta]
http://www.codeforamerica.org/governments/capabilities/open-data/
(CC-BY Code for America)

Introduction: What is open data, and why bother?

• Opendata.gov and 8 principles of open government data 

                http://opengovdata.org/

     1. Complete
All public data is made available. Public data is data that is not subject to valid privacy, security or privilege limitations.

     2. Primary
Data is as collected at the source, with the highest possible level of granularity, not in aggregate or modified forms.

     3. Timely
Data is made available as quickly as necessary to preserve the value of the data.

     4. Accessible
Data is available to the widest range of users for the widest range of purposes.

     5. Machine processable
Data is reasonably structured to allow automated processing.

     6. Non-discriminatory
Data is available to anyone, with no requirement of registration.

     7. Non-proprietary
Data is available in a format over which no entity has exclusive control.

     8. License-free
Data is not subject to any copyright, patent, trademark or trade secret regulation. Reasonable privacy, security and privilege restrictions may be allowed.


• Open Data Glossary

          https://docs.google.com/document/d/1ZbkQ2Ad66FKVj-v2T-UHKJbsh0CHV-dm9MEAoy6yT2Y/edit

Laying the Groundwork for Open Data

• Define the goals of your open data initiative and align it with organizational goals and priorities

     What to expect: How open data has worked for cities of all sizes
     -Open Data examples: Louisville, KY
     -Open Data examples: Chattanooga, TN
     -Open Data examples: Montgomery County, MD
     -Open Data examples: Pittsburgh, PA
     -Open Data examples: Albuquerque, New Mexico
     -Open Data examples: Kansas City, MO
     Read also: "Beyond Transparency" on open data's impact in various cities
     http://beyondtransparency.org/

• Build departmental support and executive buy-in

     Who needs to be at the table?
      > Executive leadership,  Internal champion, IT leader, GIS Specialist, Departmental Stakeholders

Demonstrating Value: Open data success stories

     Public Safety Open Data examples:
     -San Francisco: Crime Spotting Map
     -New York City: Targeting Illegal Building Conversions Inspections
     Economic Development Open Data examples:
     -Asheville, North Carolina: Empowering Startups
     -Charlotte, NC: Helping Local Organizations Unlock Funding
     Citizen Participation Open Data example:
     -Chicago: Flu Shot Locations
     Health and Human Services Open Data example:
     -Louisville: Restaurant Inspection Scores on Yelp
     -San Mateo County: Aggregating Community Services
     Internal Cost Savings and Efficiency Open Data examples:
     -Albuquerque: Reducing Transit-Related 311 Calls
     -Oakland: Streamlining Public Records Requests
     -Chicago: Eliminating 311 Redundancies
     Transparency and Accountability Open Data example:
     -Boston: Increasing Trust Between Government and Residents

Opening and Publishing Data

• Prioritizing data for release

     # Former Philadelphia Chief Data Officer Mark Headd recommends starting with the "Three Bs":
        Buses (transit data), Bullets (crime data), and Bucks (budget and expenditure data).

• 18 recommended datasets

     http://us-city.census.okfn.org/faq/

     1. Asset Disclosure
     2. Budget
     3. Business Listings
     4. Campaign Finance Contributions
     5. Code Enforcement Violations
     6. Construction Permits
     7. Crime
     8. Lobbyist Activity
     9. Procurement Contracts
     10. Property Assessment
     11. Property Deeds
     12. Public Buildings
     13. Restaurant Inspections
     14. Service Requests (311)
     15. Spending
     16. Transit
     17. Zoning (GIS)
     18. Web Analytics

• Compare major platform options and select an open data platform
• CKAN -- the Comprehensive Knowledge Archive Network -- is open source software powering open data platforms across the world. Provided by the Open Knowledge Foundation in the UK, CKAN is used at the local, regional, national, and international levels of government as well as in academia.
• DKAN is a Drupal-based implementation of CKAN that offers an easier installation and support burden while preserving API compatibility.
• OpenDataCatalog (ODC) is open source software originally created by Azavea for the city of Philadelphia.
• Socrata is the most popular commercial data platform provider in the United States. Socrata offers a turnkey SaaS cloud-hosted data catalog, paid for on a subscription basis. The Socrata platform includes API abilities and sitewide analytics that track consumption and engagement metrics. Socrata is used by dozens of municipal governments, including Baltimore, Austin, Chicago, Seattle, and New York City.
• 2014 Code for America Fellows compiled this "Open Data Portal Analysis" and detailed comparison which compares features and costs for some of the most common open data platform providers.

• Publish your data 

Planning for Sustainability

• Create an open data policy

     Example Policies:
     -City of South Bend Executive Order No. 2-2013
     -City of Louisville Executive Order No. 1, Series 2013
     -City of Austin Resolution No. 20111208-074
     Comprehensive list created by Sunlight Foundation:
     A Bird's Eye View Of Open Data Policies: http://sunlightfoundation.com/policy/opendatamap/

• Appoint staff to be responsible for data management

     -Chief Data Officer
     -Open Data Coordinator (ODC)

Making open data useful

• Use common open data formats

     Examples: General Transit Feed Specification (GTFS), Housefacts Specification, Local Inspector Value Entry Specification (LIVES), and other data formats

• Hold a hackathon

     Socrata's "how to plan a hackathon" doc

• Deploy apps that use open data

     Recommended apps for redeployment
     -Adopt-a-hydrant
     -Click That Hood
     -To The Trails
     -Look at Cook
     -Flu Shot Finder
     More apps can be found at Code for America Apps page
 

Open Data Field Guide by Socrata
http://www.socrata.com/open-data-field-guide-chapter/
(All rights reserved by Socrata)

0. Introduction to the Open Data Field Guide

1. Why Does My Organization Need Open Data?
-What Is Open Data?
     Is Open Data The Same As Open Government?
     Brief History of Open Data and Key Initiatives to Date
-Why Open Data? Why Now?
     A Changing 21st Century Constituency
     The Changing Nature of Government Work
     Leveraging the Community for Innovation

2. Define Clear and Measurable Goals
-Align Your Open Data Program with Your Mission and Strategic Plan
-Adapt Open Data Goals to Your Local Context
-Common Goals for Open Data Initiatives

3. Assemble a Winning Team
-The Open Data Stakeholders
-Winning Your Chief Executive’s Support
-Invite Every Department

4. Develop Your Open Data Policy
-Why Is an Open Data Policy Necessary?
     The Benefits of Good Policy
-Elements of an Effective Open Data Policy
-The Main Types of Open Data Policies
-Open Data Policy Examples and Resources
     Sample Policies and Implementation Guides
     Sample Resolution Statements

5. The Data Plan
-Which Data Should You Publish First?
     8 Steps to a Successful Data Plan
         1. Identify the data that supports your strategic goals.
         2. Adapt your open data goals to your local context.
         3. Start with the data already on your site.
         4. Analyze your site traffic.
         5. Analyze your FOIA and public information requests.
         6. Request feedback from citizens.
         7. Interview your co-workers.
         8. Don’t reinvent the wheel. Copy what works.
     What Are Open Data Leaders Publishing?
     Data Format and Open Data Standards
-Open Data Standards
-Application Programming Interfaces

6. Open Data Implementation in Six Steps
-Think About a Pilot to Start
-Phase 1 – Start Small
-Phase 2 – Get Transparency Done
-Phase 3 – Bring Developers on Board
-Phase 4 – Increase Agency Participation
-Phase 5 – Optimize for Efficiencies and Cost Savings
-Phase 6 – Federate Data with Neighboring Cities, Counties, and States

7. Engage Your Community
-Promoting Your Open Data Portal
     Examples of Success
     Engagement
-Four Essentials of Developer Evangelism
     Publish Data
     Connect With Civic Developer Organizations
     Host a Hackathon
     Be Humble
-What Apps Are Developers Building?

8. An Outstanding Citizen Experience
-Curating the Data Experience
-Rethinking the Citizen Experience
     From File Downloads to Useful Visualizations
     Say It With Maps!
     Richer Visual Context? Try Map Mashups
     The “App-ification” of Data
     Taking the Experience Mobile

9. Join the Open Data Community
-The Growing Open Data Movement
-How to Stay Connected to the Open Data Community

Acknowledgements & Glossary

Open Data Companion Kit

Project Open Data  
http://project-open-data.github.io/

1. Background
2. Definitions
2-1 Open Data Principles - The set of open data principles.
2-2 Standards, Specifications, and Formats - Standards, specifications, and formats supporting open data objectives.
2-3 Open Data Glossary - The glossary of open data terms.
2-4 Open Licenses - The definition for open licenses.
2-5 Common Core Metadata - The schema used to describe datasets, APIs, and published data at agency.gov/data.
3. Implementation Guidance
Implementation guidance for open data practices.
3-1 U.S. Government Policy on Open Data - Full text of the memorandum.
3-2 Implementation Guide - Official OMB implementation guidance for each step of implementing the policy.
3-3 Public Data Listing - The specific guidance for publishing the Open Data Catalog at the agency.gov/data page.
3-4 Frequently Asked Questions - A growing list of common questions and answers to facilitate adoption of open data projects.
3-5 Open Data Cross Priority (CAP) Goal - Information on the development of the Open Data CAP goal as required in the Open Data Executive Order.
4. Tools
This section is a list of ready-to-use solutions or tools that will help agencies jump-start their open efforts. These are real, implementable, coded solutions that were developed to significantly reduce the barrier to implementing open data at your agency. Many of these tools are hosted at Labs.Data.gov and developers are encouraged to contribute improvements to them and contribute other tools which help us implement the spirit of Project Open Data.
4-1 Database to API - Dynamically generate RESTful APIs from the contents of a database table. Provides JSON, XML, and HTML. Supports most popular databases. - Hosted
4-2 CSV to API - Dynamically generate RESTful APIs from static CSVs. Provides JSON, XML, and HTML. - Hosted
4-3 Spatial Search - A RESTful API that allows the user to query geographic entities by latitude and longitude, and extract data.
4-4 Kickstart - A WordPress plugin to help agencies kickstart their open data efforts by allowing citizens to browse existing datasets and vote for suggested priorities.
4-5 PDF Filler - PDF Filler is a RESTful service (API) to aid in the completion of existing PDF-based forms and empower web developers to use browser-based forms and modern web standards to facilitate the collection of information. - Hosted
4-6 Catalog Generator - Multi-format tool to generate and maintain agency.gov/data catalog files. - Hosted Alternative
4-7 A data.json validator can help you check compliance with the POD schema. - Hosted
4-8 Project Open Data Dashboard - A dashboard to check the status of /data and /data.json at each agency. This also includes a validator.
4-9 Data.json File Merger - Allows the easy combination of multiple data.json files from component agencies or bureaus into one combined file.
4-10 API Sandbox - Interactive API documentation systems.
4-11 CFPB Project Qu - The CFPB’s in-progress data publishing platform, created to serve public data sets.
4-12 HMDA Tools - Lightweight tools to make importing and analyzing Home Mortgage Disclosure Act data easier.
4-13 ESRI2Open - A tool which converts spatial and non-spatial data form ESRI only formats to the Open Data formats, CSV, JSON, or GeoJSON, making them more a part of the WWW ecology.
4-14 ckanext-datajson - A CKAN extension to generate agency.gov/data.json catalog files.
4-15 DKAN - An open data portal modeled on CKAN. DKAN is a stand alone Drupal distribution that allows anyone to spin up an open data portal in minutes as well as two modules, DKAN Dataset and DKAN Datastore, that can be added to existing Drupal sites to add data portal functionality to an exist Drupal site.
4-16 DataVizWiz - A Drupal module that provides a fast way to get data vizualizations online.
4-17 Esri Geoportal Server - Open source catalog supporting ISO/FGDC/DC/… metadata with mapping to DCAT to support agency.gov/data.json listings in addition to providing OGC CSW, OAI-PMH and OpenSearch. Supports automated harvesting from other open catalog sources.
4-18 Libre Information Batch Restructuring Engine - Open data conversion and API tool, created by the Office of the Chief Information Officer of the Commonwealth of Puerto Rico.
4-19 JSON-to-CSV Converter - A handy means of converting data.json files to a spreadsheet-friendly format. A similar tool can provide basic CSV-to-JSON functionality.
5. Resources
This section contains programmatic tools, resources, and/or checklists to help programs determine open data requirements.
5-1 Metadata Resources - Resources to provide guidance and assistance for each aspect of creating and maintaining agency.gov/data catalog files.
5-2 Business Case for Open Data - Overview of the benefits associated with open data.
5-3 General Workflows for Open Data Projects - A comprehensive overview of the steps involved in open data projects and their associated benefits.
5-4 Open License Examples - Potential licenses for data and content.
5-5 API Basics - Introductory resources for understanding application programming interfaces (APIs).
5-6 Data Release Safeguard Checklist - Checklist to enable the safe and secure release of data.
5-7 Digital PII Checklist - Tool to assist agencies identify personally identifiable information in data.
5-8 Applying the Open Data Policy to Federal Awards: FAQ - Frequently asked questions for contracting officers, grant professionals and the federal acquisitions community on applying the Open Data Policy to federal awards.
5-9 Example Policy Documents - Collection of memos, guidance and policy documents about open data for reference.
5-10 Example Data Hubs - Collection of department, agency, and program data sites across the federal government.
5-11 Licensing policies, principles, and resources - Some examples of how government has addressed open licensing questions.
6. Case Studies
Case studies of novel or best practices from agencies who are leading in open data help others understand the challenges and opportunities for success.
6-1 Department of Labor API Program - A department perspective on developing APIs for general use and, in particular, building the case for an ecosystem of users by developing SDKs.
6-2 Department of Transportation Enterprise Data Inventory - A review of DOT’s strategy and policy when creating a robust data inventory program.
6-3 Disaster Assistance Program Coordination - The coordinated campaign led by FEMA has integrated a successful data exchange among 16 agencies to coordinate an important public service.
6-4 Environmental Protection Agency Central Data Exchange - The agency’s data exchange provides a model for programs that seek to coordinate the flow of data among industry, state, local, and tribal entities.
6-5 FederalRegister.gov API - A core government program update that has grown into an important public service.
6-6 National Broadband Map - The National Broadband Map, a case study on open innovation for national policy. Produced by the Wilson Center.
6-7 National Renewable Energy Laboratory API program - An agency perspective on developing APIs for general use and in particular building the case for the internal re-use of the resources.
6-8 USAID Crowdsourcing to Open Data - A case study that shows how USAID invited the “crowd” to clean and geocode a USAID dataset in order to open and map the data.
6-9 Centers for Medicare & Medicaid Services (CMS) Data and Information Products - a case study of how CMS is transitioning to a data-driven culture, including the creation of a new office for information products and data analytics, the release of open data summarizing provider utilization and payment information, and the responsible disclosure of restricted use data to qualified parties.
For Developers: View all appendices (and source)
7. Open Data Engagement
Data Jam
Datapalooza
Hackathon
Online Community
FOIA Officers and Ombudsman
Templates and instructions

Open Government Data (The Book) by Joshua Tauberer
http://opengovdata.io/
(All rights reserved by Joshua Tauberer)

Civic Hacking and Government Data 
-Civic Hacking 
-History of the Movement 
-Open Government, Big Data, and Mediators
Civic Hacking By Example 
-Visualizing Metro Ridership
Why I Built GovTrack.us
Applications for Open Government
-Sunlight as a Disinfectant 
-Democratizing Legal Information 
-Informing Policy Decisions 
-Consumer Products
A Brief Legal History of Open Government Data
-Ancient Origins of Open Access to Law
-The U.S. Freedom of Information Act 
-The 21st Century: Data Policy
14 Principles of Open Government Data
-Online and Free, Primary, Timely, Accessible (Principles 1–4) 
     (1) Information is not meaningfully public if it is not available on the Internet for free.
     (2) “Primary: Primary data is data as collected at the source, with the finest possible level of granularity, not in aggregate or modified forms.”
     (3) “Timely: Data are made available as quickly as necessary to preserve the value of the data.” Data is not open if it is only shared after it is too late for it to be useful to the public.
     (4) “Accessible: Data are available to the widest range of users for the widest range of purposes.”
-Analyzable Data in Open Formats (Principles 5 and 7) 
     (5) Analyzable.
     (7) “Non-proprietary: Data are available in a format over which no entity has exclusive control.”
-No Discrimination and License-Free (Principles 6 and 8) 
     (6) “Non-discriminatory: Data are available to anyone, with no requirement of registration.”
     (8) “License-free.” Dissemination of the data is not limited by intellectual property law such as copyright, patents, or trademarks, contractual terms, or other arbitrary restrictions.
-Publishing Data with Permanence, Trust, and Provenance (Principles 9–11)
     (9) Permanent: Data should be made available at a stable Internet location indefinitely.
     (10) Safe file formats: “Government bodies publishing data online should always seek to publish using data formats that do not include executable content.”
     (11) Provenance and trust: “Published content should be digitally signed or include attestation of publication/creation date, authenticity, and integrity.”
-On The Openness Process (Public Input, Public Review, and Coordination; Principles 12–14)
     (12) Public input: The public is in the best position to determine what information technologies will be best suited for the applications the public intends to create for itself.
     (13) Public review
     (14) Interagency coordination
Data Quality: Precision, Accuracy, and Cost
Bulk Data or an API?
A Maturity Model for Prioritizing Open Government Data
Case Studies
     -U.S. Federal Open Data Policy 
     -Transparency, Participation, and Collaboration 
     -The Later Memorandums 
     -House Disbursements 
     -State Laws and the District of Columbia Code
Paradoxes in Open Government
     -The Bhoomi Program and Digital Divides
     -Unintended Consequences and the Limits of Transparency 
     -Looking for Corruption in All the Wrong Places 
     -Conclusion
Example Policy Language
     -Open Government Data Definition: The 8 Principles of Open Government Data 
     -OKF’s Open Knowledge Definition 
     -New Hampshire HB 418

Open Data Handbook by Open Knowledge Foundation
http://opendatahandbook.org/
(CC-BY Open Knowledge Foundation)

Introduction
Why Open Data? 
What is Open Data? 
How to Open up Data 
So I’ve Opened Up Some Data, Now What? 
Glossary 
Appendices

Open Data Ireland: Open Data Publication Handbook
(CC-BY Deirdre Lee, Richard Cyganiak & Stefan Decker at Insight Centre for Data Analytics, NUI Galway)
https://www.insight-centre.org/sites/default/files/publications/open-data-publication-handbook.pdf

Step-by-Step Guide to Open Data Publishing 
Step 1 Carry out a Data Audit
Step 2 Select what Data to Publish

 [Common High-Value Datasets]

Step 3 Ensure Data Protection Laws are Adhered to
Step 4 Associate Data with an Open License
Step 5 Publish Data as 3- to 5-star Open Data
     *Publish data on the Web under an Open License
     ** Publish data in a machine-readable, structured format
     *** Publish data in a non-proprietary format
     **** Use URIs to identify things, so that people can point at your stuff
     ***** Link your data to other data to provide context

[Machine-Readable and Non-Proprietary Data Formats]

Step 6 Associate Data with Standardised Metadata
Step 7 Use Data Standards
Step 8 Use Unique Identifiers
Step 9 Provide Access to the Data
Step 10 Publish Data on the National Open Data Portal

...and more resources:
"Open Government - Collaboration, Transparency, and Participation in Practice" by Daniel Lathrop and Laurel Ruma
"Open Data Guidebook" by City of Philadelphia
"Open Source for Government" by Ben Balter
"Open Government Briefing Guide" by Open Austin


Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki