Categories
ethics reflection research

Research update #60: Previously on Colin’s PhD

Looking back, it appears that it’s been 6 months since my last confession – um, post.

A few things have happened since then, less tangible progress than I would’ve liked but at the same time I feel that I’ve unraveled a few knots that had been troubling me and I’ve set the stage to get things done in 2021. (Sure, why not tempt fate, what could possibly go wrong with that)

It’s basically impossible at this point to not talk about COVID-19, as much as I’m sick of the sound of that term. It’s had a handful of notable impacts on my research, in that one of my underlying assumptions that people don’t know much about edvisors and what we do has shifted as more academics than ever have been forced to engage with us in the rapid shift to online teaching and learning. This still doesn’t necessarily mean that they have the full or correct understanding of this but overall awareness at least has changed. At the same time, the shift has also had an impact on the way edvisors work with academics at scale, so my ideas about (collaborative) working relationships need to be reframed and reconsidered.

The loss of the international students that underpinned university finances has also had a significant impact on budgets and staffing levels. People in edvisor roles have perhaps been safer than some but I have still a number of friends and colleagues that have borne the brunt of cuts and restructures and there remains a nervous instability in many institutions. Speaking more selfishly, I think this will mean that institutions will likely be less willing to share information about edvisors numbers, roles and unit structures than they might previously have been.

On a more positive note, I was fortunate to add an additional supervisor to my team, Dr Jess Frawley, who also works in an edvisor role and has been invaluable in providing some new insights into this work. One big breakthrough discussion I have had with the whole supervision team has led to me rescoping this project from my initial “boil the ocean” idea of getting insights from edvisors, academics and institutional leaders to the somewhat more manageable and realistic focus just on edvisors. (I can save the latter two for post doc work maybe – but one step at a time).

I also learnt a lot about going through the ethics process last year – doing so another two times for minor changes to my methodology. I suspect that the less detail about the nuts and bolts that is included the better. In my case, I’d initially said that I’d put out a call for survey participants and they would need to contact me before I’d give them a link to the survey. On reflection, I felt sure that this would have significantly reduced engagement so I needed to submit a modification to fix this. Live and learn.

I’m now reading a bunch of highly relevant theses that all seemed to hit Google Scholar within a few days of each other and trying to fight off the urge to radically upend all my plans for something completely new. Stay tuned for a post or two about what I’ve taken from these soon.

I finally also got around to engaging more with my PhD peers in the lab at my school. COVID-19 has probably been a blessing in that regard as it has meant that there has been much more web based activity to support this group that I’ve been able to participate in. This can be a very lonely endeavour and I really do value the conversations I have both with my TELedvisors friends and study peers.

Categories
ethics imposter syndrome reflection research

Research update #58: I’m ethical

There’s a lot about the PhD experience that I find quite daunting – I don’t think I’m alone in this – but the administration side seems particularly nerve-wracking. Getting accepted, getting my thesis proposal accepted, getting ethics clearance: they all speak directly to imposter syndrome. Most of the time this just burbles away happily in a small dark corner in the back of my mind, tempered largely by knowing that this is just part of scholarly culture and it is pretty much universal. Hearing the many many stories of others in my position through online communities and blogs like The Thesis Whisperer has been hugely helpful in understanding that this is just part of the process.

All the same, having to pass these institutional hurdles for the first time still brings it to the fore. This is when the faint nagging doubting comes into the light because there will be proof, one way or the other, that I belong or I don’t.

Happily, I do. (At least for now)

HREC came back to me on Thursday to say that they are happy with the extra information that I provided in response to their questions and it is time to move forward.

Given the current flurry of activity in universities in responding to the challenges of the COVID19 Coronavirus, I have a feeling that this may not be the optimal time for me to be asking for the time of people in roles like mine. It’s a fascinating time to be supporting learning and teaching in Higher Education in Australia, particularly given how many of our students now come from (and are still stuck in) China. Being on-the ground in institutions that are sometimes seen as slow movers when it comes to learning and teaching change and seeing how they take rapid and decisive action at scale in seriously embracing TEL is pretty exciting. There will be a lot to say and learn when the dust finally settles – whenever that might be.

Suffice to say, I’m mentally factoring in longer than normal response times for surveys and interviews. (Not that I know what the norms are anyway, but you know). At least hopefully contact uni HR teams for more generic data about numbers and titles should be less dramatic.

I can’t remember what I’ve mentioned before about my methodology but in this first phase I’ll be seeking to survey and interview edvisors about a range things relating to professional identity and perceptions of their place and value in institutions. I know essentially nothing about what to do in terms of wrangling this data and turning it into a story – I know there will be coding and Nvivo involved for the qualitative responses – but I’m looking forward to learning it.

It also occurred to me the other day that there is a great deal to be learned about what people think edvisors are and do from successful and failed job applications. The ethics around accessing the latter in particular seems like a massive swamp but it’s something I’ll think about for later.

Image by Peggy und Marco Lachmann-Anke from Pixabay

Categories
ethics methodology politics Professional staff

Research update #57: Curly questions in ethics

I heard back about my ethics application a few weeks back – it’s mostly fine but there is a big question that I need to respond to before I can go ahead. It’s essentially to do with whether the institution or individuals in the institution are the real participants.

I want to work with key informants in edvisor roles in most (ideally all) of the universities in Australia to learn about their perceptions and experiences in these roles. That’s the easy bit. I also want to gather some rich empirical data about the numbers of peoples in these roles, both in central and faculty – and other? – teams, and how these teams are structured. That’s the hard part.

The ethics committee wants to know what I am going to do in terms of getting permission from the institution to collect this data. In hindsight, this is clearly something I should have given more thought to in the research design. While to me, this data doesn’t seem particularly sensitive, there’s all manner of university politics and other sensitivities surrounding this, apparently.

My feeling is that for this data to be truly meaningful, it needs to reflect all the universities. Otherwise it is just an average or an estimate. (Which is what most of the existing research I’ve found provides.) So what happens if some institutions don’t want to share? (I don’t really expect that to be the case but people being people, who can say?)

The logistics of obtaining permission is another challenge. Am I looking at one person in the institution (maybe like a DVCA – but really I have no idea) or do I need to clear this with them and leaders in each individual faculty? Assuming 6 faculties per institution on average, 280 people? Clearly this isn’t practical.

A few things I’m going to follow up that will hopefully shed light on this. The Council of Australasian Leaders of Learning and Teaching (CAULLT) recently released a very useful environmental scan of professional learning in HE that captured exactly some of this data – though only in central teams from what I can tell. Hopefully the report’s author Kym Fraser can offer some advice on what they did in terms of permissions.

There are also some statutory reporting requirements that HE institutions in Australia have relating to reporting on staffing numbers to the government that might also demonstrate that permission isn’t needed. From what I’ve seen so far, this data doesn’t go into the level of detail that I need though and probably doesn’t go into organisational structures either. Most unis have Business Intelligence units that manage this kind of data – moreso for internal use – I’m also going to chat to them. I don’t think they will be able to make a call on permission but they may have a better idea where to go next.

Another significant question that the Ethics committee has thrown up is whether universities will have issues with their staff working as a key informant for a few hours to do work that is outside that person’s ordinary duties. I really have no answer to this – though I kind of wonder if this question would have been asked if it was academic staff that I was planning to work with. (I probably won’t say that in my response.) It does bring me back to the seeking permission question/dilemma.

Have you had any experience with these kinds of questions? Got any tips?

Categories
education ethics methodology PhD research Uncategorized

Thoughts on: “Doing Insider Research in Universities” (Trowler, 2012) Part 3 – Good research design and ethics/politics

I’m not sure that this is how I’ll process all of the books that I read – in fact I’m almost certain that it isn’t – but I’ll continue this series of posts about Trowler – Doing Insider Research in Universities because I have found it to be a great way to dip my toe into the many issues that I will face in my research.

The next two chapters look at value and robustness in insider research (which, again, I take to be about being able to defend your methodology – and choosing a good one) and then the ethics and politics of insider research in your university, which is pretty much unavoidable.

He opens with a discussion of some of the criticisms that case studies face as well as some of the responses to these. I’ll leave it here in full as it sums it up well.

Case study researchers may find their work subject to the following criticisms (Flyvbjerg, 2006): it only yields concrete, practical knowledge rather than the supposedly more valuable general, theoretical and propositional knowledge;  generalization from one case is not possible and so the research does not contribute to the development of knowledge; the research can generate hypotheses, but other methods are necessary to test them and build theory; case study design contains a bias toward verification, i.e., a tendency to confirm the researcher’s preconceived notions. Both Flyvbjerg and Yin (2009) refute these criticisms, but those who research their own universities need to be clear about precisely what their research questions are, what the rationale behind the research design is, and what the truth claims are. This advice holds for any kind of research, but other designs tend to draw less critical fire.

(He also highlights Gomm et al (2000), Simons (2009) and Yin (2009) as great starting points for further investigation of case study methodology)

Trowler dips back into some of the ontological questions that he touched on earlier in the book, comparing the merits of the true vs the useful. (I may be oversimplifying this). This draws on Sayer’s notion of “practical adequacy” in prioritising the usefulness of information. I kind of get it but think I’ll need to dig a little deeper. I can see how some true things mightn’t necessarily be valuable but as for things that are kind of true…?

This is echoed in further discussions of Bassey’s idea of “fuzzy generalizations”. In short, this is about acknowledging that life is complex and theory won’t always accommodate the range of factors at play. So that rather than saying that in situation A, if B happens then C will follow, we might say in situation A, if B happens then C will generally follow between D and E% of the time. It’s not as neat and arguably not as helpful but no doubt more realistic.

In terms of the design of research, Trowler posits twelve questions for researchers to consider to test the rigour and quality of their proposed methods.

1. In designing the research, how do I know that my planned approach will answer the questions I have more fully than any other?
2. How do I design the research to take best advantage of the benefits of insider research while avoiding its pitfalls as far as possible?
3. Conceptually, how do I represent my organization, its culture and its practices? (And how does this representation shape my design?)
4. How and from whom will I secure access to the data I need? (Why them and why not others?)
5. Whom should I inform about the project, and how should I describe it, when I seek ‘informed’ consent? (And how might this affect my data?)
6. How will I ensure that the project is run ethically so that all participants and institutional bodies are protected? (While at the same time being as transparent as possible to readers so they can judge the robustness of my approach and conclusions?)
7. If I am using participant observation, what are the ethical issues in relation to the people I observe in natural settings? (And how might my answer to that question affect my data?)
8. If using interviews, what measures should I take to deal with interview bias? (And will they assure a sufficient degree of robustness?)
9. What should the balance be between collecting naturalistic data and formally ‘collected’ data? (And how can I offer assurances of robustness about conclusions drawn from both?)
10. How should I analyse the different forms of data I have, given that there will almost certainly be a large amount of various sorts? (And how do I ensure that sufficient and appropriate weight is given to each form of data in generating conclusions?)
11. How, and how much, will I communicate my findings to participants to ensure that they are happy with what I intend to make public? (And will this affect the way I present my conclusions to other audiences?)
12. Generally, in what other ways can I satisfy the reader about the robustness of my research and its findings?

(I was a little hesitant to just paste this in holus bolus but it all seems particularly valuable).

In very pragmatic terms, there will also inevitably be a number of ethical and political issues to consider when undertaking insider research in one’s own institution. The question of whether to anonymise the institution and the research participants is a live one – though at this stage, to me, it seems impractical and counterproductive, particularly when it could mean reducing the number of sources of data for fear of not being able to correctly reference them. The lack of transparency could also arguably lessen a reader’s view of the robustness of the research.

I also think that the question of to what extent people change their behaviour under observation is a valid one, however there are no doubt ways to mitigate this.

Politically, senior leaders in the organisation will imaginably want to feel confident that the research won’t damage the reputation of the university before granting access. Does this then lead to self-censorship and selective reporting if there are areas where there is room for improvement?

At a higher level of ethical debate, the selection of standpoints from which to pose questions and to begin observations and investigations can raise some concerns. If I fail to incorporate the perspective of voices that are less often heard, am I guilty of perpetuating the status quo?

Lots to think about and to be perfectly blunt, I would be naive not to factor in the question of what asking the wrong person the wrong question might mean for my career prospects in the organisation. Fortunately I have a little time to think about some of these things.

 

Trowler, P. (2014). Doing insider research in universities.
Simons, H. (2009). Case study research in practice. SAGE. http://www.dawsonera.com/depp/reader/protected/external/AbstractView/S9781446205365
Bamber, V., Trowler, P., & Saunders, M. (2009). Enhancing learning, teaching, assessment and curriculum in higher education theory, cases, practices. McGraw-Hill : Society for Research into Higher Education & Open University Press. http://public.eblib.com/choice/publicfullrecord.aspx?p=771392