Using machine learning to analyse radio content in Uganda

Pilots

Radio and community radio in particular are critically important medium for vulnerable groups in Uganda. In the 2014 Uganda Population and Housing Census, most households reported radio as their main source of information.


Pulse Lab Kampala and partners have conducted case studies which provide evidence of the fact that public radio discussion includes reports of local incidents and first-hand experiences reported by citizens that are not gathered with other quantitative or qualitative assessment methods and are not formally recorded anywhere else. The pilots show how the exploration of the unconstrained public radio discussion can inform programmes to achieve the SDGs. And they also prove that the radio application developed by Pulse Lab Kampala and partners can be used to capture and analyse in a systematic way these discussions.

The Pilots

Overall the pilots provide evidence of the fact that public radio discussion includes reports of local incidents and first-hand experiences reported by citizens that are not gathered with other quantitative or qualitative assessment methods and are not formally recorded anywhere else. The pilots show how the exploration of the unconstrained public radio discussion can inform programmes to achieve the SDGs. And they also prove that the radio application developed by Pulse Lab Kampala and partners can be used to capture and analyse in a systematic way these discussions.

Analysis Results

Click the circles to view results of the different pilots

Partners:
Resident Coordinator Office
Office of the Prime Minister
UNDP Uganda
Ministry of Health Uganda
Save the Children Uganda
Ministry of Health Uganda

Early warning and influx of refugees in Uganda

We analysed people’s public discussions about refugees as the crisis started and high influx of refugees fled into Uganda in mid July 2016. The pilot was conducted to “take the pulse” of the South Sudan refugee crisis based on people’s voices as expressed publically on radio in Uganda. While the data analysed is not representative of the overall universe of analysis, valuable insights on the unfolding crisis were found. Big data provided in this case anecdotal evidence of emerging issues relevant to feed early warning systems.

Figure 1 – Main topics of conversation related to refugees mid-July to mid-August 2016.

Figure 2 – Emerging vulnerabilities related to refuges from November to December 2016.

Figure 3. Early warning on disease outbreak in August 2016.

Examples of radio discussion:

“About 400 elephants and so on are just seated every day [daily] in the sub counties of Lapono and Paimol so foods like millet, sorghum and whatnot, the elephants are doing what, spoiling them all.”

“The father of the chair [Chairman] Local council XXX sub county has given a threat [has threatened] and has given a go ahead to the youths to start giving poison to the elephants that are destroying their places. The words of that elder [The Chairman] came out after two days when some elephants that are suspected to have come from South Sudan went and attacked Abalo Kodi village and destroyed food about 20 acres that belongs to the people in the gardens.“

(Various discussions in October 2016/ Grassroots+Media / Northern region)

“The father of the chair [Chairman] Local council Three of Atiak sub county has given a threat [has threatened ] and has given a go ahead to the youths to start giving poison to the elephants that are destroying their places. The words of that elder [The Chairman] came out after two days when some elephants that are suspected to have come from South Sudan went and attacked Abalo Kodi village and destroyed food about 20 acres.”

(October 2016 / Media / Northern region)


Recording losses associated to local disasters

Pulse Lab Kampala has conducted this pilot in collaboration with the National Emergency Coordination and Operations Centre (NECOC) in the Office of the Prime Minister (OPM), District Disaster Management Committees and UNDP.

The objective of this pilot was to establish whether small-scale disasters are being reported on the radio, whether these reports contain the information (location, type of disaster and losses) required to complete the Desinventar database, and whether these local reports provide additional information on the ways natural hazards impact people’s lives and livelihoods.

Reports on small scale disasters

The pilot proves that small-scale disasters are reported on a regular basis on radio and that these reports contain the information on location, extent of damage and type of disaster. Some of these disasters are very under-reported as lightening, small-scale bushfires and local hailstorms and in these cases, even anecdotal evidence of local reports are extremely useful for planning purposes.

Examples of radio discussion:

G1: I am here in Ssemuto. (( only famine)) We planted our beans, Maize all of them have died but we hear that the rains that side of yours in the city and yet for you you don't farm. God needs to have some mercy and he rains the rain this side in our villages. M1: The one you are angry at as they never get angry at him. He will cane you. You will feel it.

(September 2016/ Grassroots /Central region)

The heavy rains with hail stones have destroyed 13 buildings in Kampeka and Ssemuto subcounties that are in Nakaseke district, which has left hundreds and hundreds without homes and has affected them as well. The rain that fell so much has also destroyed many roofs of buildings and among them are 3 school buildings that were left without roofs. 2 Health centers as well were destroyed completely. In addition, things planted in the garden [crops] about 100 acres in total [were destroyed] in the two sub counties to the extent that you look at them with pity. The schools that have been destroyed are Semuto secondary school, Tuzunkere primary school and Erina memorial primary school. Kikandwa health center two and Lukumumbi Health center two are among the hospitals that have been destroyed.

(August 2016/ Media /Northern region)


Local Governance and public health service delivery

The objective of the pilot was to evaluate the availability of public radio content to analyse perceptions on the quality of health service delivery that could be used to inform local governance processes. The objective of the analysis was to better understand public discourse around public health service delivery in Uganda to inform strategies to address bottlenecks in the quality of health service delivery at local level.

The pilot was conducted in dialogue with the Department of Political Science at Gothenburg University, UNDP and the Ministry of Health of Uganda.

Unfiltered citizen’s reports

The analysis does demonstrate that there is a high volume of public radio discussion in the topic of health service delivery on an ongoing basis. Most of the conversations are complains and reports about the poor quality of public services. The figure below shows the results on to what and to whom is the poor quality attributed to. Note that the size of the bubbles indicate the volume of discussion per category identified.

Figure 4 - Topics of radio discussions identified related to healthcare service delivery.

The following are some examples of the radio discussions

When you go to the hospital they tell us ‘go to the clinic of so and so’…. Most of the doctors now own clinics, and if possible the government should prohibit doctors working in government hospitals from having clinics. There is a lot of medicine theft by the doctors and they take the medicine to their private clinics.

(August 2016/ Grassroots /Northern region)

So for that reason you leaders of the government [government workers] work hard and bring for us a big hospital in this very place in Lakang. So it will help people because moving from here and going up to Lacor [main hospital in Gulu], the kilometers are many. Malaria starts [attacks] with the child and running up to Amuru center [making it to the hospital is difficult], you find that the child dies in the night.

(September 2016/ Grassroots /Central region)


Radio campaign monitoring in real time

Development partners in the region traditionally develop and conduct radio campaigns to implement behavioural change campaigns. The issues addressed in these campaigns might vary from water and sanitation, to gender violence or HIV/AIDS. the costs of such campaigns is significant monitoring and evaluation is challenging and would ordinarily require that someone conducts monitoring visits to the locations where the radio stations involved in the campaign are located.

Pulse Lab Kampala has conducted this pilot with the NGO Save the Children which has started a global campaign to promote that 15 million of the world's excluded children have access to life-saving healthcare and quality education. As part of the campaign, the song called “Remember Me” will be aired in Kampala, Gulu and Moroto from July 2016.

Results

Software was trained to recognise the song, and applied to the recording stream for Gulu (Mega FM and Radio King), and Kampala (Capital FM). Evaluation of the performance of this software was done by assessing each of the detections it made during the evaluation period, and having an analyst listen to the corresponding recording to confirm whether the campaign song was in fact being played, i.e. whether each was a true detection or a false alarm. Evaluation was also done by means of an analyst listening through radio broadcasts to manually identify times that the campaign song was played, and then testing how many of those plays were also picked up by the software.

StationTotal playsNumber detected Number missedFalse alarms
Mega FM (Gulu)302190
Capital FM (Kampala)4220
The table above shows results of automated campaign detection.


Finding insights to fight malaria

The pilot aims to identify talk related to the incidence of malaria that might provide insights for the implementation of malaria eradication programmes.

Results

Although some discussion of malaria-related issues were identified, this case study did not uncover a substantial amount of new insight. The filtering for malaria discussion was done by screening discussion including the word “malaria” and also the ways in which the disease is referred to in local languages, e.g. “omusujja” in Luganda (literally “fever”). However, the word “malaria” seemed to be used infrequently, and the local language terms are not specific to malaria, and can be used to refer to any febrile disease.

The following are some examples of the radio discussions

And this type of medicine that is being sprayed, that medicine [insecticide] for mosquitoes. We have not yet found out at all its benefit to us because when they do not spray the medicine, there are no mosquitoes, but once they spray there are so many mosquitoes that come. We need to be explained to about that because all these things are going to kill us, the Acholi people, and in the near future we shall be finished.

(Grassroots/ Northern region)

Malaria is now so bad at the moment. They don’t even treat the patients who are so ill they are being carried in on bicycles. Sometimes you can be kept at the hospital from 7am till 6pm without getting medicine.

(July 2016/ Grassroots/ Northern region)


Data Privacy and Data Protection

A project involving analysis of people’s voices expressed on the radio naturally requires high levels of privacy considerations and protection. Even though radio is a public forum, Pulse Lab Kampala takes a number of extra measures to protect the people behind the data and applies strict privacy principles to ensure that individuals are not exposed in a way that is to their disadvantage. Pulse Lab Kampala has used tools developed by UN Global Pulse and partners to address data privacy and data protection issues to conduct the pilots including the data privacy and data protection principles and the Privacy Impact Assessment (PIA) tool.

By anonymization of data, limited access to the data and destruction of the data when the intended analysis has been performed, individual’s personal space and integrity is protected and respected.

About Pulse Lab Kampala

Pulse Lab Kampala is the third lab of the UN Global Pulse network under the leadership of the UN Resident Coordinator. It supports the UN Country Team and the Government of Uganda to achieve the Global Goals for sustainable development. As a regional hub, it leverages data innovation, new sources of digital 'Big Data' and real-time analysis techniques.

Sign up for our monthly data update here.