Investigators' Blog
NSSI (Not Such a Silly Idea… but do it properly) #5
Welcome back to Not Such a Silly Idea! In this exciting new season, two summer students continue to critique a government document, and this time they have interactive visualisations! In case you missed the first part of our epic journey, you can read instalments one, two, three and four.
One of the claims made in the National Statement of Science Investment (NSSI) which we wanted to look into was based on this graph (Figure 1) on page 20 of the NSSI:
Figure 1
The NSSI asserts that the government’s investment in science has “increased by over 70% since 2007/08”, and that “our science has improved in response”. This is followed with several graphs which show things like increasing publication rate, increasing number of papers in top journals, growing research workforce, and increasing international collaboration over time. These graphs each stand on their own, but fail to relate these improvements to the amount of money the government has been spending on science. One of our goals was to re-visualise these graphs in a way that clearly showed a correlation (or not) with increasing government investment, and we will address that later on. But before that, we had to investigate this data on government spending.
We remain puzzled that the NSSI claims an “over 70%” increase in government expenditure in the last eight years, when according to their own data the increase is more than 80%. Self-deprecation aside, when we went back to The Treasury we discovered that the graph on page 20 is not adjusted for inflation. This immediately indicated that the increase in spending was not quite as significant as claimed, since the government would have had to up their investment by about 40% just to compensate for the devaluing dollar. Using the Reserve Bank calculator, we found that government spending on science has actually increased 55% (not 82%) since 2007/08 and 46% (not 87%) since 2004/05.
After adjustment the government’s spending still showed a rise, however, so we started looking for the implications of that rise. We created the graph below to see whether the government might be able to claim New Zealand’s burgeoning number of publications as correlated with its financial support. To make this graph we had to include a time lag. Cash pays for research, but it takes time for that research to be published. We weren’t sure, though, how long that lag is on average. We did a regression analysis of our data using this equation, in case you’re interested:
We used a range of time lags between investment and publication, from no lag to four years. The time lag that showed the strongest correlation was two years. (We have far too little data to infer anything universal from this – it is simply our best guess.)
And voila, there is a positive correlation between the amount of money poured into ‘science and innovation’ and the number of papers churned out. But what does the money do? Do scientists publish more frequently when they are better funded? In other words, does greater funding increase productivity measured in papers per full-time researcher? Or does the money go towards increasing the number of people producing papers in the first place?
Six years ago, Professor Shaun Hendy published this paper (page 56). It drew the conclusion that although our publication output increased hugely from 1990-2008, that increased output was due to a rise in the number of researchers, not the number of papers each researcher produces in a year. Having read this paper, we expected to find similar results for 2000-2014, but we were surprised to see that both FTEs and productivity have been steadily on the rise, according to both OECD and Statistics New Zealand data.
The publication output numbers we retrieved from SciVal, and we found two different sets of researcher full-time equivalents (FTE) data; one in the Main Science and Technology Indicators database on OECD.Stat, and the other in the R&D surveys on Statistics NZ. There was a confusing discrepancy between these sources because the latter breaks down Higher Education researcher FTEs into ‘researcher’ and ‘student researcher’, while OECD.Stat makes no distinction, and the numbers didn’t come to the same total. Our best guess is that one counts only PhD students, while the other also includes Masters.
These two graphs are very interesting, because in spite of the differences, they support that both the number of science researchers and their productivity has increased.
So, apart from the fact that MBIE needs to be careful with accurately presenting information, we can conclude government investment in science has indeed increased, and that it is correlated with increased output of publications, increased research workforce, and increased productivity. Of course, just from these graphs we can’t be sure which way round the causal relationship works. A great incentive, surely, for both parties to keep up the good work!
Win tickets to Adam Rutherford ‘Gene Genie’ tour
WIN TICKETS TO Dr Adam Rutherford’s Royal Society of New Zealand ‘Gene Genie’ tour!
Follow Te Pūnaha Matatini on Twitter or Facebook and tell us your #sciencehero for your chance to WIN tickets to the below events.
GENE GENIE TOUR
Te Pūnaha Matatini is proud to support the Royal Society New Zealand tour of Dr Adam Rutherford, geneticist, BBC 4 science presenter and science writer – touring 15-22 March 2016.
Dr Adam Rutherford is a science writer and broadcaster. On radio, he is the presenter of BBC Radio 4’s flagship science programme, Inside Science, as well as many documentaries, on the inheritance of intelligence, on MMR and autism, human evolution, astronomy and art, science and cinema, scientific fraud, and the evolution of sex.
TOUR DATES
Christchurch | Genetics and disease 6pm Tuesday 15 March
Aurora Centre, cnr Greers Road and Memorial Ave, Burnside High School, Christchurch
Dr Adam Rutherford discusses the genetics underlying disease with Professor Vicky Cameron (Christchurch Heart Institute), Professor Nigel French (Massey University) and Professor Parry Guilford (University of Otago)
Dunedin | Conservation genomics 7.30pm Thursday 17 March
St David Lecture Theatre, cnr St David and Castle Sts, University of Otago, Dunedin
Dr Adam Rutherford discusses ancient DNA and conservation genomics with University of Otago researchers Dr Catherine Collins, Professor Neil Gemmell and Dr Michael Knapp
Tauranga | Family and genealogy 7pm Monday 21 March (refreshments served from 6.30pm)
Tauranga Yacht Club, Sulphur Point, Tauranga
Dr Adam Rutherford discusses genealogy and family issues with University of Otago Professors Lisa Matisoo-Smith, Stephen Robertson and Hamish Spencer
Auckland | Genetics and health 6pm Tuesday 22 March
Auckland Museum Events Centre, The Domain, Parnell, Auckland
Dr Adam Rutherford discusses medical genetics and epigenetics with University of Auckland and Auckland DHB researchers Dr Don Love, Dr Rinki Murphy and Professor Cristin Print
Buy tickets>
Dr Rutherford is also touring as part of the New Zealand Festival, Wellington
*Feature image: Adam Rutherford Royal Society of New Zealand.
Competition:
– Runs 02 March to 21 March, 2016.
– Social media competitions run through Te Pūnaha Matatini Twitter and Facebook
– Each winner receives two tickets to selected event
– A maximum of up to 40 tickets available (up to 10 per event)
– Winners selected based on best entries for each social media event
– Winners selected at Te Pūnaha Matatini’s discretion.
The First Post
By Ben Curran
It’s an interesting thing, writing the first post. It’s an interesting thing writing the first line.
Whether it’s the first line of a paper, a chapter, a grant application or a blog post, I always find the first line … daunting. It’s only now, having finished the thesis and other things need to be written, that I recall how awful that first line is. Even if there is a specific goal behind the writing, an idea that you set out to communicate, what words do you choose for the first sentence? Who are you talking to? What sort of tone are you after? These are things that paralyse the first line.
And then there’s the times when you’re forcing yourself to write when there is no specific goal other than to practice writing. I liken this situation to the one I encountered all too often in one of my previous incarnations as a bartender – there’s always a customer who comes in at some point and says “surprise me”. Most often they got a glass of water.
Like it or not, writing is a large part of what we do. Sure, the thinking, the testing, the figuring out what’s going on are important, but in the end they mean nothing if we can’t communicate the results. And for larger audiences, writing is the primary means of communication.
Writing has to start somewhere though. Writing the first line, whether it’s a good sentence or not, is always awful. It is almost certainly going to be at least edited, if not entirely removed. Which makes it, in the greater scheme of things, not particularly important. This, to a certain extent, can be extended to the entire first draft of pretty much any work. One of my PhD supervisors, in an effort to get me writing, used to stress that whatever I wrote for the first draft was going to come back with red ink all over it. I was told to just write something, anything, a foundation upon which the story you are trying to tell could be built.
If you’re not used to working with wood, there is often a feeling of trepidation in making the first cut. Making the first bend in a piece of metal, applying the soldering iron for the first time to a circuit board. All of these things impart a sense of beginning and often the thought that runs through your head is “what if I screw it up”. It’s the same thing with writing. Measure twice, cut once, Dad said. The first draft is only the first measurement. The first sentence is only the first line on the plans, drawn with pencil.
So if you have a specific idea to communicate, start writing. After a while that feeling of trepidation is replaced by familiarity. Knowing the first draft is only the rough plan of your work means that, eventually, writing the first sentence becomes … an odd thing. Just odd. And yet familiar, interesting even.
And as a scientist, when I see something interesting, I usually want to stop and take a serious look. Turn it over, see how it works. This is where it can be good if you don’t have a specific idea to communicate, put that first, odd sentence down and see where it takes you. Possibly somewhere very much like here.
Video Workshop for Scientists
Wellington – Wednesday March 30 / Auckland – Thursday 31 March 2016
The Science Media Centre is hosting a workshop on communicating science in 90 second videos aimed at an online audience and leveraging platforms like Youtube and Vimeo.
Great short videos can be produced using the high-definition camera built into your smartphone or tablet. How can you harness this technology to bring your science to life? What are the best ways to shoot, edit and distribute your video content?
This workshop will answer all of your questions and introduce you to great tools that will help you in the video production process.
Limited to 20 places. Participants will be selected based on the video concepts outlined in the application form.
Workshop topics:
- Why is video so powerful?
- Scripting and storyboarding short videos
- Finding images and footage
- DIY video – harnessing your smartphone to make great videos
- Software and tools you can use.
Applications close 15 March
Course fees: FREE to attend — by invitation
Find out more and APPLY NOW!
Conference: Multicore World 2016
Conference Summary
15 – 17 February 2016
Shed 6, Queens Wharf – Wellington, New Zealand
Bringing together business and technical know-how of developing and deploying multicore and manycore based solutions (5th edition). Including panels and debates on: “HPC in the Cloud: the SKA example”and “TOPS: The Open Parallel Stack”
Multicore World is a global think-tank that allows you to network and learn about all aspects of multicore and manycore technology and business. Organised by Open Parallel since 2012, takes the pulse of this fast growing ecosystem: sessions cover latest developments in software and hardware, applications and the businesses behind these trends.
Find out about:
• Software for the enterprise in a complex world
• Machine Understanding: the future of the Data Centre
• Big Data and Internet of Things: scalability and industry applications
• Heterogeneous Computing and Exascale Computing
• TOPS: The OS for a Data Centre Rack Scale Computer
Browse through the program> openparallel.com/multicore-world-2016/program-2016
Conference Summary
Conference Schedule
Discount offer
25% discount on tickets purchased by February 12. Email danene.jones@auckland.ac.nz to find out how, or check out the February 10 newsletter.
Te Pūnaha Matatini Cycle Challenge
What: Te Pūnaha Matatini Cycle Challenge – join the team and commit to reducing your climate impact.
When: February 1 to 29
Where: From wherever you are across Aotearoa
Something wheely good is happening at Te Pūnaha Matatini HQ – Director Shaun Hendy bought a bicycle and Executive Manager Kate Hannah is riding to work! As a result, the whole HQ team has committed to reducing our climate impact by starting a February bike challenge.
You can take part by using the #tpmride hashtag on Twitter. Post us your ride to work photos and tell us how many kilometres you’ve clocked up.
Auckland riders> sign up to the Te Pūnaha Matatini Auckland Bike Challenge team and log all your rides.
Tips for reducing your climate impact
Check out this Motu post that features research by Suzi Kerr, a Motu senior fellow and TPM principal investigator, on tips about how Kiwis can reduce their climate impacts.
NSSI (Not Such a Silly Idea… but do it properly) #4
By Catherine Webb and Nicola Gujer
A long time ago in a galaxy far, far away, we started deconstructing the National Statement of Science Investment and considering the claims it makes.
Our first blog post ended with an unanswered question – is the proportion of a country’s papers in the global top 10% of citations a good way (or at least the best available way) to measure science excellence? The NSSI’s definition of excellent science is “well-designed, well-performed, well-reported research, recognised as such, e.g. through peer review.”
In preparation for our first blog post, we were dealing with a graph that suggested New Zealand produces science which is not quite as ‘excellent’ as our comparative small advanced economies, because a lower percentage of our publications make it into the 10% most cited publications worldwide. In that post, we discussed the limitations of that graph, but nonetheless concluded that it does show New Zealand lagging slightly in the production of ‘top-shelf science’.
Still, we were curious to see whether there are any more useful ways of measuring ‘science excellence’ that might paint a different picture.
What if just looking at the papers with the most citations is not the right approach? Are citations a good way to measure excellence at all? One flaw with citations is that papers can become highly cited for two reasons: by being amazing, or by being such poor science that everyone wants to correct it (like Wolfe-Simon’s paper on a bacterium that thrives on arsenic). Also, as we discussed in our second post, citations tend to follow a ‘rich-get-richer’ power-law distribution, which makes a well-known paper garner even more citations, while another paper, nearly as good, can dwell in obscurity all its life.
However, even if citations are not a great way, they may currently be the least-bad way of assessing the impact of publications. But what kind of citations should we use to compare countries? Every country has different areas of specialty, and if they are alone in that area, they may not be cited very much by other countries, even if their science is top-notch. New Zealand, for example, is environmentally isolated and unique. Our conservation or agricultural papers for instance may not be of immediate relevance to anyone else as much as they are to us. If our science is rather useful to ourselves, but not to the rest of the world – should that make it less ‘excellent’?
We broke the data down into only intranational citations per publication and only international.
Because international citations make up the vast majority of any small country’s citations, these have the greatest impact on the percentage of publications in the 10% most cited. Thus, in terms of ranking countries, these two measures of ‘excellence’ can be roughly used as proxy.
Does New Zealand’s rate of intranational citations balance our lagging overall ranking?
| Country | Intranational citations per publication | International citations per publication |
| New Zealand | 1.25 | 5.95 |
| Denmark | 1.70 | 8.35 |
| Finland | 1.46 | 6.87 |
| Ireland | 1.13 | 7.05 |
| Israel | 1.23 | 6.98 |
| Singapore | 1.35 | 7.82 |
| Mean average | 1.35 | 7.17 |
It’s possible that New Zealand produces publications which are more relevant and therefore cited more within our own country than they are in other countries; we just don’t cite enough to pull up our average very far, but this is conjecture.
In any case, does New Zealand do well enough by intranational citations to let us off the hook of general lack-of-excellence? Well, we certainly have room for improvement. The next question is, obviously, how to improve – a subject for another article, where we examine government investment and its effect on the science sector.
Look out for our next episode in the New Year, and may the force be with you over the holidays!
*Featured image by XKCD comics.
Call for nominations for Software Carpentry Instructor Training
New Zealand eScience Infrastructure (NeSI) is pleased to announce the offering of a Software Carpentry Instructor Training workshop at the University of Auckland, 28-29 January 2016 and would like to invite candidates from Te Pūnaha Matatini. This workshop kicks off our 2016 training programme, which is quickly followed by Research Bazaar 2016 (ResBaz), and presentations by key international training leaders at eResearch NZ 2016.
Software Carpentry is an international initiative to upskill researchers, in research computing skills to help them be more productive. It is usually delivered in the form of a two-day highly-interactive workshop, and has seen great successes worldwide.
NeSI, the national leader in High Performance Computing, is an official affiliate of Software Carpentry Foundation, and in partnership with several institutions across the sector we have delivered a series of Software Carpentry workshops nationwide during 2015. We are seeing great demand for research computing training in New Zealand’s research community, with many events sold out within a few days.
We are fortunate to have Aleksandra Pawlik from Software Sustainability Institute, UK, here to run a *two-day* intensive workshop to train and certify new instructors. This workshop will cover the basics of educational psychology and instructional design, and look at how to use these ideas to create an effective learning experience.
New certified instructors from your institution will then be authorised to offer Software Carpentry workshops to the local research community on a regular basis, and NeSI will continue to support and sponsor future training offerings at your institution (our sponsorship in the main comes from our membership, which covers the administrative fee otherwise payable to the Foundation).
The Instructor Training workshop is limited to 20 participants. We ask for at most two candidates per institution for the selection process. We expect to confirm selected candidates, and inform them, before the new year.
There are number of considerations to keep in mind when choosing candidates for this workshop:
- A technical/academic staff member or a Ph.D student who is technically capable and experienced in scientific computing will be an ideal candidate.
- Once candidates have been nominated, we will direct them to complete a brief online pre-selection survey which will allow us to select the final group of 20.
If selected, attendees are required to complete a number of tasks prior to attending the Instructor Training workshop.
- Newly trained Instructors must commit to running/teaching at 2 workshops within 8 months upon completing instructor training, with the first workshop within 3 months.
These requirements are in place so together we can build an effective team of local Instructors committed to putting their skills into practice.
Nominate candidates by emailing georgina.rae@nesi.org.nz with “Call for nominations for Software Carpentry Instructor Training” in the subject line before Wednesday December 16th – include the names and contact emails for your candidates.
Our December Hui
By Dr Rebecca Ford
Last week I attended the Te Pūnaha Matatini Investigator Hui in Christchurch. While this is not the first time the group of academic investigators have come together at such events since the launch of the new Centre of Research Excellence in February 2015, it was my first time meeting the full team of Investigators and Whānau since joining Te Pūnaha Matatini earlier this year. So, on Thursday 3rd December, a group of over 50 academics and students descended on Canterbury University (who graciously hosted our rowdy crew – thank you!) for two days filled of intellectual stimulation, innovation, networking, debate, fun prizes, and good food and beer!
Although I came to the hui with some ideas about the work being conducted by Te Pūnaha Matatini investigators, I left with a much richer appreciation of what makes it pretty unique in today’s world and the opportunities afforded by being part of Te Pūnaha Matatini.
I have been an academic (post PhD) for nearly 5 years and one of the main issues I have noticed during this time is that, as academics, we are often constrained in our thinking and research by the disciplinary boundaries within which we are employed and evaluated. These boundaries are artificially constructed – nature does not operate within disciplines – and can be troublesome when trying to tackle some of the key environmental, social, and economical problems we’re seeing in the world today.
Interdisciplinary and integrative research is absolutely vital if we are to better understand and guide socio-technical and socio-ecological transitions toward more sustainable futures, and a key part of Te Pūnaha Matatini’s uniqueness is that the academics involved have interests and expertise spanning various aspects of the environment, economy, and society – from knowledge and innovation in business to the evolution of the universe, from environmental management to bed-bugs. And more so, these creative minds are actively seeking out conversations, research topics, and methodologies that span the traditional disciplinary boundaries that so many shy away from.
In my own research I am lucky enough to work with an interdisciplinary research team, inclusive of engineers, computer scientists, physicists, sociologists, psychologists, economists, and modellers. Since our research program kicked off just over three years ago, I have observed our team go through the four stages of development – forming (transitioning from individual to team member), storming (intra-team conflict and resistance of self-change), norming (acceptance of team norms, personal roles, and idiosyncrasies of fellow members), and finally performing (diagnosing and solving problems, making decisions).
All too frequently teams try to rush through the non-productive form, storm, and norm stages. While this may be a seductive idea to get to the performing stage more quickly, it is ultimately dysfunctional. Groups, just like people, need time to develop and find maturity to tackle complex problems. The Investigator Hui allowed Te Pūnaha Matatini’s teams to share concerns, ideas, and expectations; providing the space needed to develop trust, and gain individual and interpersonal insights – key building blocks to establishing highly performing research groups. And while perhaps not much work was directly achieved, the time spent together engaged in team conversations, celebrating each other in the award ceremony, drinking and eating, and learning about patent data, sexism in science, and bed-bugs in New Zealand Department of Conservation huts has served to strengthen our relationships, provide an opportunity to network, and enabled future planning; ultimately creating the space for the magic to happen.
As I reflect back on our two days in Christchurch, I am grateful to be part of such an engaged, interesting, and open community; Te Pūnaha Matatini really is ‘the meeting place of many faces’ from many backgrounds and with many interests, and I look forward to observing and contributing to the unfolding future of the Centre, working together to tackle some of New Zealand’s (and the world’s) complex problems across environmental, societal, and economical issues.
NSSI (Not Such a Silly Idea… but do it properly) #3
By Catherine Webb and Nicola Gujer
The adventures in data-sleuthing continue. In this blog series, two summer students examine the National Statement of Science Investment 2015-2025, and appraise its use of data. So far we have found one case of partially-correct-yet-misleading-ness, and another of axis-labelling-deficit. This time we take a look at Academic-Corporate Collaboration.
On page 17, the NSSI makes an intriguing statement: “Only 3.2 per cent of New Zealand publications have academic and corporate affiliations, suggesting scope for more collaboration.” We asked, where did this information come from? By what standard do we have ‘scope for more collaboration’? And is that a good standard to use?
Implicit in that statement is the assumption that an abundance of academic-corporate collaboration is a good thing – a claim to be investigated later on.
Firstly, we found that the “3.2%” used in the NSSI is found under ‘Academic-Corporate Collaboration in New Zealand’ on SciVal. The first thing we noticed was that this statistic does change over time, although it fluctuates less than field-weight citations. From the time it was retrieved for the NSSI published in October and the time we checked it in November, it had already fallen to 2.6% (that’s around a 19% drop). Hence, we wanted to check out how stable this measure is over a longer period of time.
We didn’t find anything very remarkable in that regard: on average, the academic-corporate collaboration rate of each small advanced economy deviated about 17% from its initial value over five years, with New Zealand squarely on the mean (+16.7%).
This also helps us answer the second question, ‘compared to what do we lack collaboration?’ The graph shows how our nation’s academic-corporate collaboration measures up to that of other small advanced economies (SAEs); Denmark, Finland, Ireland, Israel and Singapore (widely accepted as comparative countries due to similarities in factors that affect science performance). Using the same measure, this is the data as it stands in Nov/Dec 2015:
Percentage of publications with both academic and corporate affiliations:
| New Zealand | 2.6% |
| Denmark | 4.8% |
| Finland | 2.5% |
| Ireland | 2.7% |
| Israel | 3.3% |
| Singapore | 2.0% |
| Mean average | 3.0% |
We see that by this standard, NZ is below average, but not markedly. We are still above Singapore and Finland, and with our ‘3.2%’ measured earlier in the year, we would have been above the present average!
Presumably, when the NSSI claims that New Zealand is lacking collaboration, they are using the small advanced economies as a reference – they cannot be referring to the worldwide percentage of academic-corporate collaboration, as that is only 1.34%. And yet, if they are comparing us to other SAEs, we are near the average and certainly not significantly underperforming.
Finally, however, we found a significant problem with the New Zealand statistics on SciVal. Academic-corporate collaboration is defined as at least one academic institution and one corporate (business) institution being affiliated with the same publication. On SciVal we found evidence that Crown Research Institutes (which are government entities and not private businesses) are being counted as corporate organisations. Here is one example of a paper listed as an academic-corporate collaboration:
As you can see, the only contributors to this paper are universities and two Crown Research Institutes; AgResearch and Landcare Research. Although our Crown Research Institutes have been ‘corporatised’, meaning that they are managed much like businesses, New Zealand is unique in this respect. Many countries have government science research organisations – the equivalent of our CRIs – but which are treated as purely public government institutes, such as the CSIRO in Australia. This presents an obstacle in drawing conclusions from this data set: comparing academic-corporate collaborations between countries is problematic when New Zealand calls corporate what other countries call government.
This inclusion of CRIs as corporations is skewing the total stats on collaboration in New Zealand, but by how much it is difficult to tell. Unfortunately, it is not possible to find the collaboration data adjusted to exclude CRIs on SciVal. CRIs cannot be excluded from the affiliation search without excluding papers with genuine collaboration between universities and corporations. SciVal’s lack of Boolean operators makes more nuanced searches impractical. Thus, we cannot provide a more correct number for New Zealand’s academic-corporate collaboration percentage than that published in the NSSI. But what we can say is that the NSSI’s number is not accurate, and when CRIs are excluded, NZ’s true academic-corporate collaboration percentage should be, in fact, lower than the NSSI reports.
We have to trust that a similar mistake has not been made in SciVal’s database for any other of the small advanced economies. Without a better dataset we cannot draw any conclusions about the potential for improving academic-corporate collaboration in New Zealand. If anything, this project has highlighted the need for comprehensive data-keeping, as well as taking care in how it is used.
*Featured image by XKCD comics.