Investigators' Blog

Te Pūnaha Matatini was able to offer New Zealand university students paid 10-week internship positions over the 2018-19 summer, which involved working on projects with various partner organisations.

Out of a total of 160 undergrad and postgrad students from across the country who applied for the programme, 29 were successful and placed at 11 partner organisations – including government ministries, iwi organisations and private companies.

Although the students had a wide range of backgrounds, they all had one thing in common – a passion to change the world through data and its applications and contexts.

“This programme provides students with invaluable data analytics experience and new perspectives while working for organisations in a real-world setting,” said Dr Alex James, Te Pūnaha Matatini’s Deputy Director of Industry and Stakeholder Engagement.

“Through its student internship programme, Te Pūnaha Matatini is able to engage with partners to complete small-scale projects with defined outcomes, develop relationship networks, and introduce talented students to potential employers.”

“This programme provides students with invaluable data analytics experience and new perspectives while working for organisations in a real-world setting.”

Now in its fourth year of operation, the 2018-19 programme featured an excellent variety of projects, and the overall feedback from both students and industry stakeholders was very positive.

Streamlining the process of social network analysis at AgResearch

University of Canterbury Masters of Applied Data Science student Romalee Amolic (pictured above) spent the summer in Hamilton working with scientists at AgResearch, helping develop a tool to streamline survey data for social network analysis.

In addition to providing a much-needed tool for AgResearch, Romalee is now working out how to commercialise her software so more people can take advantage of it.

“From our perspective, we got exactly what we’d hoped for, which was a new viewpoint and different expertise from what we may have normally recruited,” said Helen Percy of AgResearch, Romalee’s supervisor.

Vision Mātauranga in partnership with Ngāti Whātua Ōrakei

Brianne Halbert (left) and Megan Leijh (right), who are both at the University of Auckland, worked together on a project that is part of an ongoing research partnership between Ngāti Whātua Ōrākei and Te Pūnaha Matatini.

Led by their supervisor Kate Hannah, Te Pūnaha Matatini’s Deputy Director, Equity and Diversity, and Dion O’Neale, a Te Pūnaha Matatini Principal Investigator, the ‘He waka eke noa’ project combines qualitative and quantitative methodologies to work with iwi and hapū data, centralising Māori data sovereignty.

An important goal of our internship programme is to have students with complementary skill sets working together. That was certainly the case for Bri and Megan. Bri is undertaking a double major in Computer and Data Science, while Megan is completing a conjoint Law (Hons) and Arts degree in political philosophy, law and politics.

“While these disciplines may appear vastly different, we were able to find a lot of overlap and even harmony in our exploration of inclusive education for Māori,” wrote Bri and Megan in a subsequent co-authored blog.

“From the outset, a major goal was to utilise our respective disciplines for research while keeping the essence of te Ao Māori alive throughout. Thus, we incorporated kupu o te wiki, watched Te Kaea and participated in a lot of korero”.

Working with Te Hiku Media to improve access to Te Reo Māori

Bachelor of Engineering in Mechatronics at the University of Canterbury, Cherie Vasta (pictured above), worked with both Te Hiku Media and Dragonfly Data Science on a project to aid in the development of a Māori voice assistant.

The tool is designed to make Te Reo Māori more accessible and fun in the digital age At the completion of Cherie’s internship, the project was documented and all the code uploaded online to allow other developers at Te Hiku Media to progress it further and demonstrate the abilities of the Rāpere box.

“I got a great feeling of accomplishment from my work,” said Cherie. “I’m grateful to Te Pūnaha Matatini for connecting me with Te Hiku Media and providing me with the opportunity to have this internship.”

Further details about individual projects

Following the completion of their placements, several interns were invited to blog about their experiences on the Te Pūnaha Matatini website.

Read more about the work done by Romalee, Cherie, Brianne and Megan.

Improving education for all is as uncontentious as political clauses get here in Aotearoa, championing this issue is on the front burner of nearly every government body, institution, and organisation across the country. Compare this to the endless debate, controversy, resistance, and even hostility which meets attempts to address protocol and resolve for our Māori students.

He waka eke noa and ‘Auckland 2050’

This year’s research project at Te Pūnaha Matatini, ‘He waka eke noa’, is part of an ongoing research partnership with Ngāti Whātua Ōrākei that aims to highlight some features of our education system in Aotearoa—chiefly those affecting Māori—as part of Auckland Council’s greater 30-year vision, ‘Auckland 2050’.

He waka eke noa follows in the wake of last year’s research, ‘Analysis of Well-Being’, which endeavoured to interpret open responses of 684 registered hapū members, taken from surveys encompassing various forms of codified knowledge and kōrero in the hopes of directing the development of a hapū well-being framework. Well-being was measured with regards to life satisfaction, education, housing, proficiency and abilities in te reo and, more generally, te Ao Māori. Last year’s well-being research proposal was awarded Marsden funding through the Ministry of Business, Innovation and Employment’s (MBIE) Te Pūnaha Hihiko: Vision Mātauranga Capability Fund, which invests in ‘the development of skilled people and organisations undertaking, research that supports the themes and outcomes of our Vision Mātauranga policy.’

A major component of our research cannot be revealed here for privacy reasons, but for the purpose of sharing some of the motivations and ways we engaged with our research, we can look at the educational climate of Aotearoa as a whole, indigenous ways of knowing, and the shared stake we have as a community to improve mainstream education for our Māori students.

Using data collected by the Ministry of Education, we constructed the below graphs to show 2017 post-secondary completions by subject. We found that Māori students were either largely underrepresented or overrepresented across major subjects. Further, we saw more spread in subjects taken amongst other ethnicities while Māori students predominantly stayed in familiar domains—especially at earlier levels of study.

However, there seems to be more spread in subjects once Māori students reach a Bachelors level. This may suggest continuation in studies can affect the way in which Māori students engage with subjects- branching out into different fields after acquiring confidence in mainstream education institutions.

Research in keeping with te Ao Māori perspectives 

Owing to our shared identities of being both Māori Pākehā, as well as students- we attribute a lot of strength throughout this project in our ability to appreciate the unique issues and challenges faced by students with shared membership. We also acknowledge the capabilities for dynamism and resilience. From the outset, a major goal was to utilise our respective disciplines for research while keeping the essence of te Ao Māori perspectives alive throughout. Our team engaged in quantitative analysis as well as discourse analysis pulling from a multitude of various texts that laminated on indigenous taxonomy and concepts surrounding Mātauranga Māori.

Mātauranga Māori embodies a complex network of codified systems of knowledge transfer and storage to which universal constructs are framed in both past and present, existing and non-existing. Cognisance and assessment of information and protocol envelops moteatea (chants, poems), whaikorero (oratory, speechmaking), maramataka (calendar), waiata (songs), pepeha (quotations), whakataukī/whakatauki (proverbs), whakapapa (genealogies) and pūrākau (stories)—each with its own categories, style, complex patterns and characteristics’ (Lee J, 2008).

This culturally-embedded system founded on kanohi kitea (face-to-face) interactions between individuals, whānau, hapū, and iwi is an all-encompassing body of knowledge based on evidence, culture, values and worldview. Despite being a rich form of gathering and sharing information, these traditional ways of knowing have often been considered incompatible with local pools of thought and sanctioned as illegitimate in the wider scientific and academic communities.

In rumination of Aotearoa’s current educational climate, it’s hard to imagine that the above mentioned would not play a significant role in how Māori students interact with mainstream constructs of knowledge as well as the capacities of education providers, peers, and the public to appropriately assess and acknowledge learning systems so undivulged in the domains to which they usually operate. We might also consider how the role of whānau can be affected in the enormous task of accessing and providing tools relevant to outside industries of learning and their abilities to advocate on behalf of their tamariki far from the sources of support provided within their everyday communities. As Māori student education figures indicate improvements in academic participation, the alarmingly high SSEE rates (stand-downs, suspensions, exclusions and expulsions)— at nearly all levels of study— only corroborates the narrative that the current education system is failing. The system in place is robbing its students of having an equal chance to prosper in falling short of its most essential obligations– to encourage and inspire student potential. The damages placed on already strained communities to which these students are part of seems to fuel a never-ending cycle that sets up generation after generation with even less opportunities to thrive.

Unfortunately, the displacement of Māori is not consigned to the past. Historical attempts to keep te Ao Māori outside of the local mainstream includes legislation such as the Tohunga Suppression Act, the Education Ordinance Act and the Native Schools Act, and a nationwide ban on te Reo Māori, including ‘A wide range of punishments used against children who speak te Reo at school (including corporal punishment)’.

Amid cultural asphyxiation, Māori risk losing their voice, abilities to navigate on their own terms, and essentially – their mana. Even in moments of advocacy we are limited in our capacities. We look at issues surrounding Māori and the needs for address but from a Māori standpoint, there is a need for redress. An overwhelming loss of trust in the mainstream educational institutions prevails when we miss these opportunities of knowing.

Conclusion

In conclusion, institutions cannot remain mutually exclusive in inclusive learning environments. There is a demand for institutes for Māori and Pākehā across Aotearoa, to foster a respectful, understanding and empathetic community. Culture is a learned system. Access to Māori systems of support in schools will elevate New Zealand children’s comprehension of how culture, language, and heritage empower their own identities. The immersion of these two systems of education will lead to a broader sentiment of cultural heritage will encourage New Zealand’s future communities to practise more tolerance and acceptance of cultural diversity.

He waka eke noa in a literal sense translates to the ‘canoe which we are all in without exception’. For the purposes of this project, we might attribute this whakataukī to a collective consciousness and gentle reminder that, when we are in a waka, there is unity in a shared purpose. Here, we look to education.

Authors

Brianne (Bri) Halbert and Megan Liejh are students at the University of Auckland. Bri is pursuing a double major in Computer and Data Science, and Megan is completing a conjoint Law (Hons) and Arts Degree in Political Philosophy Law and Politics. While these disciplines may appear vastly different, they were able to find a lot of overlap and even harmony in their exploration of inclusive education for Māori.

Reference

Lee J. 2008, Ako: Pūrākau of Māori teachers’ work in secondary schools. [Unpublished PhD thesis]. Auckland, New Zealand: University of Auckland.

The Archaeology and Physics Departments at the University of Auckland, as well as contributors from other universities, have been collecting data on obsidian artefacts from the north part of New Zealand. To date, this project has data on over 2,500 such artefacts, obtained from various sources including historical studies done on obsidian to more recent studies done by current archaeologists at the University of Auckland. Part of the aim of this research is to look at “Social Network Analysis of Obsidian Artefacts and Māori Interaction in Northern Aotearoa New Zealand” which is the title of a recent publication which involved my Te Pūnaha Matatini and industry supervisors.

Why study obsidian?

Obsidian is a volcanic glass which is found at several locations in New Zealand. It is hard and brittle such that when a piece is broken off (called a flake), it has sharp edges. This made it very useful as a cutting tool in pre-European New Zealand. By analysing the elemental compounds of the artefacts, it can be determined where each artefact was sourced. By comparing this to which archaeological site each artefact was found at, my supervisor Dr Dion O’Neale has been able to infer social networks of pre-European New Zealand. Dion analysed the geographical least cost paths and found that distance was not always the main factor in determining where each archaeological site sourced its obsidian flakes from. Therefore, by analysing obsidian artefacts, a lot of information can be gained and it is the aim of this research project to be able to infer this type of information and even more regarding pre-European Aotearoa New Zealand.

With so much varied data the need arose to have a central data infrastructure where all the various data records can be stored along with protocols to support data quality and provenance. This data needed to be accessible by various parties from various departments and universities.

The main steps I took to complete my internship project included:

• Choosing and learning to use an appropriate database software
• Schema design
• Data cleaning
• Scripting for automated data uploading

These steps were not necessarily sequential and often ran in conjunction with each other. For example, since there was a variety of data sources, while I was doing the data cleaning I came across new data fields in which case I had to edit the schema to reflect the new field. However while doing the data cleaning, I often came across discrepancies or unknown variables in the data which I needed to wait to hear back from other people about before I could proceed.

It surprised me how long it took me to design the schema. Data cleaning often takes the longest amount of time. In some sense the data cleaning did take some time because while I was designing the schema, I was also figuring out what data to keep and what not to keep. This greatly reduced the time it took for me to clean and format all the data tables to be ready for upload. After that, finding and learning to use an appropriate database platform also took a while. Finally, writing the scripts for automatic uploading to the database took a couple of weeks.

Author

Kate is currently studying for a Master of Applied Data Science (MADS) at the University of Canterbury. 

For 10 weeks over the 2018-19 summer, I was involved in a project with Te Hiku Media and Dragonfly Data Science to aid in the development of a Māori voice assistant. The motivation for this project was to make Te Reo Māori more accessible and fun in the digital age.

During my internship I achieved the creation of a “box” called Rapere (translation of “Raspberry” into Te Reo Māori) containing a Raspberry Pi computer which is connected to the internet, some lights, a speaker and a microphone. This box has been coded to be continuously listening for spoken voice, and when this is detected it records what is being said until there is a longer break in the speaking (this file is overwritten each time a recording is made).

Cherie’s recording equipment set-up.

The recording is transcribed using Te Hiku Media’s Application Program Interface on koreromaori.io. The transcription that is returned to the box is compared to some key words which mean the speaker is likely asking to hear the news or to listen to the radio, or to stop playing. If these are heard, then the news or radio stream is played or stopped, and otherwise it goes back to listening for these phrases. The phrase “kia ora” lights up an LED for a few seconds. The box is able to listen for commands while playing audio, which allows the user to stop audio playing. The project was documented and all the code uploaded online to allow other developers at Te Hiku Media to progress it further and demonstrate the abilities of the Rapere box.

I experienced a great feeling of accomplishment from my work with Te Hiku Media and Dragonfly Data Science. Going from a bunch of components and an empty raspberry pi computer to having a working program with two different APIs and which plays the news on my correctly saying the appropriate phrase in Te Reo was more than I thought I would be able to achieve and I am proud of what I achieved with the help of my supervisors. I am grateful to Te Pūnaha Matatini for providing me with the opportunity to have this internship.

Author

Cherie Vasta is a student at the University of Canterbury who is going into her final year of a Bachelor of Engineering in Mechatronics. Cherie enjoys problem solving and working as a part of a team. She is excited by working with high-end technologies, as she would like to be at the forefront of engineering the future.

Romalee Amolic talks about her 2018-19 Te Pūnaha Matatini Summer Internship with AgResearch where she worked on a project to enhance social network analyses of biosecurity information in the New Zealand tourism industry, so that such analyses can be conducted faster and more effectively in the future.

Social Network Analysis (SNA) is a powerful data analysis technique which often helps in identifying hidden relationships and other critical information in a communication network.  Data for an SNA can be collected from various sources which may result in extensive pre-processing and cleaning time as compared to the time needed for actual network analysis. Hence, this project aimed to use data carpentry to streamline the use of social research data (e.g. collected through surveys) to be able to conduct social network analyses quicker and more effectively in the future.

Better Border Biosecurity case study

The Better Border Biosecurity project, a multi-partner, cooperative science collaboration which analyses the exchange of biosecurity information in the New Zealand tourism industry, is used as a case study to develop and test methods which streamline the SNA process. The data for this project includes 154 responses from tourism providers across New Zealand who named up to 3 sources from whom they seek or receive biosecurity information. Information about the location and role of the respondents was collected. Some additional questions were also included in the survey such as the form and frequency of communication, the usefulness of information and the trust between parties.

This information was then used to perform Social Network Analysis in Gephi – a powerful interactive social network analysis tool. However, the survey data had to be first converted into a format fit for network analysis. The conventional approach for cleaning the data is discussed below.

Conventional approach

The survey responses were collected in Excel sheets. The data pre-processing and cleaning was done manually using Excel.

Problems with the conventional approach

• It involved dealing with the data manually which was an extremely time-consuming process and needed about 1-3 weeks depending on the complexity of the data.
• The process was prone to human errors which reduced the potential of the data.
• It required skilled labour for an extended amount of time and hence, increased the costs involved.
• It led to data inconsistencies.

Hence, taking all these problems into consideration, a generic automated process was developed to clean the data as discussed in the following sections.

Data Cleaning and Pre-processing

In a network, each node represents a unique identity. Hence, the most important task in cleaning the data was to recognise and remove inconsistencies in the names which occurred due to the textual nature of responses. The following techniques were used to clean the data in Python:

• Initially, all the names were made lowercase for the analysis.
• Special characters were removed.
• Rows containing missing or no information were removed.
• Trailing or unnecessary white spaces were removed.
• Incorrect spellings were identified and removed using a spellchecker. The challenge here was to differentiate between the proper nouns (such as ‘EcoZip’) and dictionary words (such as ‘Adventures’). For example, an entity name ‘EcoZip Adventures’ was misspelled as ‘EcoZip Adventres’. A conventional spell checker would consider ‘EcoZip’ as a spelling error along with ‘Adventres’ as both the words are not found in the dictionary. Hence, a solution was developed to distinguish the proper nouns from actual dictionary words in entity names and correct spelling errors in the data.

• A custom algorithm was developed to identify abbreviations in the text and replace it with the full name. e.g. ‘DOC’ was identified as ‘Department of Conservation’.
• Several names which were written similarly but were however, the same entity, were identified and merged. This is the most significant part of the process or the most “satisfying” part, as a user described it. An example is shown below:

• All the names in the network were also compared pairwise to further remove any inconsistencies and generate a list of consistent and unique names involved in the biosecurity information exchange.

This cleaning process reduced the entities from 319 (including inconsistencies) to 139 consistent and unique entities (nodes) in the network with 335 relationships (edges) between them which were then used to generate visualisations.

Social Network Analysis

Directed maps were generated using Gephi which were then further analysed. An anonymised example of one of the social network maps generated is shown below:

An example of anonymised social network map generated using Gephi.

The information obtained through this network analysis can now be used by biosecurity providers to better target information exchange within the New Zealand tourism industry.

Achievements

1. This application significantly reduced the time (from 1-3 weeks to a few hours or minutes) in cleaning and pre-processing the data before analysis.
2. As a result, the costs involved in the conventional extensive processes, which involved a lot of manual effort, were also reduced.
3. The new streamlined process almost eliminated the human errors involved in the manual inspection of data.

Conclusion

Hence, through this case study, an application was developed, which streamlines and automates all the steps starting from loading and cleaning the data up to the generation of data sheets to be used in the SNA. Although, this is currently a Python application, the development of a GUI based interactive SNA application design is currently under consideration.

I would like to acknowledge Helen Percy, my industry supervisor and Penny Payne, the social scientist at AgResearch for their invaluable support during this project.

Author

Romalee Amolic is a Master of Applied Data Science student at the University of Canterbury. In February 2018, she completed her summer internship with AgResearch, Hamilton. She thoroughly enjoyed her internship project which involved streamlining and increasing the efficiency of the data cleaning and network map generation processes at AgResearch. She is passionate about harnessing the power of data analytics to improve the lives of people. She eagerly looks forward to applying the skills learnt, in fulfilling her aspiration of becoming a data scientist.

Romalee (centre) with Helen Percy, her industry supervisor (right), and Penny Payne, the social scientist at AgResearch (left).