Don’t you worry, Harry. You’ll learn fast enough. Everyone starts at the beginning at Hogwarts, you’ll be just fine.
Chapter 1: Data is magic
As you walk through the halls of your new school, facts about the world float all around you. The people who walk through the school corridors, that live in the paintings, biographies of the authors of the books that line the walls, through the window you see numerous plants and animals and can identify them all.
Whether you are an experienced wizard, able to hiss strange commands in Python and other programming languages or a lowly muggle, contributing to Free Knowledge through other skills, you have come to the right place: Wikidata.
Wikidata is a free and open knowledge website holding over 25 million pages of facts, each representing a part of the world, a person, a place, a piece of art, a book. Wikidata provides structured data for Wikipedia and other Wikimedia projects as well as a growing number of websites and apps. Wikidata is built by a community of volunteers working in almost 300 different languages, just like Wikipedia.
Data shared in Wikidata becomes free knowledge, released into the public domain, available for everyone to use, share and build upon. Many organisations are choosing to make their knowledge available to the whole world through Wikidata. Galleries are adding information about their paintings, libraries their book catalogues, archives their historical records, museums their objects.
Chapter 2: Fantastic graphs and where to find them
Collating data from many sources into one place allows us to analyse and visualise it, helping us to understand many aspects of the world in a new way. In the same way Wikipedia helps us understand the world through words and images, Wikidata helps us understand the world through data. It allows us to explore great works of fiction…
the relationships between them…
A family tree visualisation of Harry Potter characters, as visualised from Wikidata. Screenshot via Wikidata: Family Trees.
and the groups they belong to…
A treemap of groups in the Harry Potter universe. Visualisation created via Wikidata, public domain/CC0.
The visualisations that aid our understanding of the world will only grow richer and more complex as as more data is added to Wikidata and more tools are created to use it.
Chapter 3: Wikidata as an enchanted map
The stories created by J.K Rowling in the Harry Potter books have sold over 400 million copies and translated into over 70 languages. They have been a major part of millions of children’s (and adults’) lives and played a major part in their journey of learning and loving to read.
Literacy is a fundamental requirement to access information, formal and informal education, employment and our ability to address global challenges. But how do we understand the state of literacy and education globally? And how can we understand the scale and the complexities of the barriers preventing access to education?
Many organisations are collecting data on state education across the world and the Sustainable Development Goals provide us with targets to work towards. Wikidata and Wikipedia allow us to collate and share this information with a huge worldwide audience of 500 million people who use the sites each month.
The information on Wikidata is collected from many sources, from books, journals and newspapers to databases and videos. This newspaper article about the shortage of teachers worldwide includes the fact that there are 263 million children worldwide who are not in school. It provides a broad understanding of the number of out of school children but no detail or nuance. It does not tell us about children’s access to education in different countries, by income level or how it is changing over time. To find this information we have to go back to the source used by the newspaper, a dataset on children’s access to education, produced by UNESCO.
By adding all the data from this dataset to Wikidata we can provide a more complete understanding of the state of education. We can go even further and combine it with related data from other organisations to provide an even more complete picture of education worldwide. Having a large amount of high quality data on Wikidata will allow it to be used by researchers as well as for a general audience.
Heatmap of items on Wikidata around the world. Visualisation by Addshore, public domain/CC0.
Chapter 4: Magic is hard
How do we achieve this? Adding individual facts to Wikidata is often a simple process but adding the thousands, tens of thousands or even hundreds of thousands of facts in a large dataset manually is not practical. Instead we must use special software to import the data en masse which presents its own set of challenges.
There are some tools and resources available to import data into Wikidata, but it is a complicated process requiring many different skills and steps which are often not well documented, making it something of a dark art. Non technical people can see the value in adding datasets to Wikidata but lack the technical ability to make it happen. This can leave us feeling like muggles and seeing the people able to import data like wizards.
Chapter 5: Muggles are magic too
Adding datasets to Wikidata requires more skills and knowledge than simply being able to import the information. For Wikidata to be a rich and accurate source of information we need to find a way to draw in the expertise of different people. This includes those with a deep understanding of the subject, a knowledge of where to find reliable datasets and the subtle differences between different kinds of information.
Both muggles and wizards need to be able to work together, each contributing their own skills, experience and specialist knowledge in the same way Wikipedia articles are created, with the input of 10s or even 100s of people.
To help this to happen we have created a workflow and documentation to make it easier for people to collaborate on importing datasets. These pages allow contributors to pool their knowledge and skills, breaking the process of importing a dataset into smaller tasks and providing a way to learn the skills needed to complete each step. The pages allow the Wikidata community to make decisions together, helping people to learn from previous data imports and progress in their technical skills and turn into wizards.
A central place to request, organise and record the import of datasets, the process is broken into stages requiring different skills.
A practical guide to learn how to import datasets into Wikidata which integrates with the existing Bot Requests page.
A place to request that prepared data is imported into Wikidata using existing tools like Quick Statements and Mix n’ Match.
A page to discuss partnerships with organisations and work on data imports.
A page to give for potential partner organisations an overview of Wikidata including the benefits of adding data and visualisation tools available.
Records which datasets have been imported into Wikidata, important for understanding what data is in Wikidata and keeping data from external data sources up to date.
Chapter 6: Collaborative spells
Now that we have these resources we can each contribute our skills and knowledge to help import data into Wikidata:
If you are new to Wikidata there is a huge amount you can contribute, some of the most important tasks on Wikidata require no technical knowledge.
- Suggest datasets: One of the most important tasks is to identify data that can be imported into Wikidata, this could be from an organisation you work for or simply a subject you are interested in. To suggest a dataset go to the Data Import Hub, then simply create an account and fill in the form on the page.
- Help robots import data: Not all data import can be done automagically, datasets being added to Wikidata can often include some data that is already in Wikidata, the Mix n’ Match tool allows us to examine and compare data, making suggestions of possible matches with existing items.
- Translate: If you are able to speak other languages please translate the pages so that other language communities can work together to import data.
Tip: This page can help you understand the terms used on Wikidata and how it works.
Wizards in training
Some people, like experienced Wikipedia contributors, have some technical knowledge but they are missing some of the skills they need to fully import a dataset themselves.
- Work with muggles: help new users learn about Wikidata and how to contribute to it.
- Learn wizard skills: unlike in the Harry Potter universe, in Wikidata muggles can, with enough time on their hands, become wizards themselves. Now that the data import process has been broken up into stages you don’t need to learn all the skills at once. E.g here is a video to learn how to query Wikidata, an important stage in learning to create visualisations
These new pages allow you to work more easily to collaborate with your muggle friends to import datasets. You can do things that take you a very short amount of time that take Muggles days but they can also do much of the manual work that takes so much time. Using the Data Import Hub to document the datasets you are importing, it will allow experts to provide information on the datasets and document what data is being added to Wikidata.
Chapter 7: Wikidata as Neville Longbottom
Neville Longbottom begins his time at Hogwarts as a shy and quiet young student with few skills and abilities, by the end of the series he leads Dumbledore’s Army, destroying the final Horcrux (a giant snake) with the Sword of Gryffindor. Wikidata is young and developing and people may not see the potential of Wikidata yet. It can only achieve its goals using the different skills and knowledge of each member of the community. These new pages are early steps to make it easier and simpler to add and maintain data in Wikidata. They need input from everyone to grow and improve.
Differences of habit and language are nothing at all if our aims are identical and our hearts are open.
John Cummings, Wikimedian-in-residence, UNESCO
Jens Ohlig, Wikimedia Germany (Deutschland)
Navino Evans, Histropedia
Special thanks to Martin Poulter and Adam Shorland for their assistance with Wikidata queries.