For the First Time Ever HSE Students Participate in the ‘Student Experience in the Research University’ Project

Interest in international education ranking is growing all over the world, and universities are trying their best to take higher positions. At the same time, the objectivity of the global rankings increasingly gives rise to doubts. Igor Chirikov, Director of the HSE Center of Internal Monitoring, tells HSE news service what they take into account and what they don’t, and what the alternatives are.

— What can supplement the global university rankings, such as the Times World University Ranking or the Shanghai Ranking?

— One of the most popular attempts to overcome the rankings’ limitations is the international project ‘Student Experience in the Research University’. It was started by the University of California – Berkeley in the early 2000s, and today this project is run as a consortium of leading research universities all over the world – in the USA, Europe, Brazil, China, and South Africa. Recently the Higher School of Economics has become the first Russian university to join the consortium.

SERU is not a ranking, but a comparison of topical indicators of educational process in universities around the globe. The comparison is carried out based on an annual survey among students from all consortium member universities. In the questionnaires students provide many details on the process of their study: about how they distribute their time, how they interact with teachers, about various forms of work implemented at the university. The survey takes place every year in spring, and all results are processed and published in a digital database where member universities can see them. Actually, HSE students have recently received the questionnaire.

It is remarkable that the student comparison is carried out not by universities, but by education programmes. This means that we can compare education processes for economics students in Oxford and the HSE.

— Why do students participate in the survey?

— Participation in the survey is a great opportunity to compare your life at the HSE with the lives students in leading universities of the world feel they have. In addition to that, the questionnaire is a way of attracting the HSE management’s attention to the problems faced by students. And finally, all survey participants can have a go at the lottery and win HSE branded souvenirs or the special prize – a tablet PC.

— Which university’s students will fill in the questionnaire?

— The questionnaire will be filled in simultaneously in leading universities all over the world, by students at the University of California – Berkeley in the USA, Oxford in Great Britain, and Amsterdam University College in the Netherlands, for example. 

— What are the benefits of this approach for analyzing the situation and evaluating the universities?

— The main advantage is that the project allows us to understand how the education process works in a specific university today. Students’ answers give quite a full picture of what types of activity are seen to be part of studying. In addition to that, it is for the first time that researchers directly address students: until now those compiling rankings weren’t bothered to find out what students think.

Finally, unlike the global rankings, this project promotes experience exchange and cooperation. The universities aren’t competing against each other but comparing each other’s experience in order to improve their education programmes. As far as I know, many consortium member universities use the results of SERU in strategic decision-making.

 

Dr. John Aubrey Douglass

Dr. John Aubrey Douglass, Senior Research Fellow - Public Policy and Higher Education at the Center for Studies in Higher Education (CSHE) at the University of California – Berkeley; SERU-International Principal Investigator gave a special interview to Igor Chiriikov.

— John, can you tell us a little about the history and the background of the international consortium “Student Experience in the Research University”?

The project began around about the year 2000 when a colleague and I convened a group of faculty and institutional researchers at the Center for Studies in Higher Education  to explore the idea of a system-wide survey of University of California students. We concluded that there was a need to create a single and more in-depth survey of undergraduate students. We needed  to know more about who our students are and their level of academic and civic engagement. We knew that there are a great variety of students’ experiences -- it’s different for everyone at a place like Berkeley, there are many communities, and the disciplines are really important. 

Then around 2008 we decided to expand the project to include other selective top research intensive universities in the US. We approached members of the Association of American Universities (AAU) which is a group of the top-tier institutions in the US and Canada, both public and private. Discipline based SERU data is the most important benchmark of how you’re doing.

SERU-AAU has now grown to include over 15 universities, plus the nine campuses of the University of California. We continue to attract and have interest from other public AAU institutions. So far, we  have not been so successful in bringing the privates into the fold, with the exception of the University of Southern California. There are a lot of reasons for that but we hope to soon recruit a few more top-tier private universities.

For a long time in that process I’ve been engaged in international comparative research. I saw that this similar data and research on the undergraduate experience was an important requirement internationally. So we basically took the same model: we approached top-tier research institutions as determined largely by international rankings, but also how important they are in the regional context; we asked, do they have the capacity to take on a project like this; is there an interest by the leadership of the institution, so that it’s not a project buried somewhere in a department? And so, a year and a half ago we launched the pilot that included three Chinese Universities: Hunan University, Nanjing University and Xian Jiaotong University; the University of Campinas (or Unicamp) in Brazil, the Amsterdam University College which is a joint venture of the University of Amsterdam and Vrije University of Amsterdam, and the University of Cape Town in South Africa.  We have nearly completed the SERU-International pilot. Among the questions we are exploring is whether there are big cultural differences among our international members, and are there similar interests and concerns among these institutions?

We are really grateful to have National Research University Higher School of Economics in Moscow joining, as you perfectly fit all of those criteria. The other new member is the University of Bristol, which we were very thrilled about, because it is among the top-five institutions in the UK.

— There are some other initiatives to launch surveys on student engagement, both in the US and internationally. One of them is the National Survey of Student Engagement (NSSE), housed at Indiana University. What’s the difference between these two projects?

NSSE is an important and very large survey operation based at Indiana University and their model is to provide a standardized survey instrument for all types of colleges and universities in the U.S.  The relationship with the institutions is largely a contractual one: ‘here is the survey, we’ll do this survey for you, here is the data, and that’s it’. There are no larger efforts to share data, do collaborative research, or to share best practices.

In the early stages of SERU, we considered NSSE and some of the members still do NSSE in off years, because there are some good aspects to that survey. But early on our group in the University of California System realized that that instrument was not specifically designed for the research university environment. We started with a different model in mind. First, this would be a census survey, because we need a wide range of data to be able to drill down to the discipline level of various subgroups of students. Second, SERU was to be an online survey – a model that is more cost-efficient with better ways of getting and sharing data. And third, members will help shape survey content and have the ability to customize it to meet local needs and collaborate with other SERU members on issues they care about. So, none of those pieces were in NSSE’s design. They are making some modifications over time, but it is still a kind of contractual relationship with universities that employ their services. It’s not like our model that is collaborative, with a survey instrument that isoriented towards research university environment. 

Another important part of the Consortium goes beyond the survey itself – we have opportunities for our members to get together and share data under certain protocols, to collaborate on research topics, and to share best practices. So the survey instrument is of course a kind of foundation of the project but it is much, much more than simply a survey.

— That’s fascinating. And if to sum up what are the main goals of the Consortium?

The major goal is for each SERU member to be more systematic in developing and generating data on the experience and behavior of their students, and to know more about who their students are and the challenges they face in the course of their education. Because, frankly, many universities, especially large universities, do not really know much about their students -- their socio-economic background, their interests and the problems they face. So that is one aspect.

Dr. John Aubrey Douglass: “A Tremendous Window for us to Gauge what Is Going on Globally in Higher Education”

One way I like to talk about it is to say: there has been so much attention on the faculty side of the equation for what a research university is supposed to do. And this has been said many times over a long period, that there are three general functions of a research university: teaching, research, and the third is public service – how an institution interacts and local communities and economies.  Almost all of the attention has been on the faculty side of this equation, and you can see that in rankings. Where are students in that equation? Students are more than just receivers in terms of teaching. This gets to the notion that the best universities are building and sustaining a scholarly community with various players in it, it’s not just simply handing out information to students and saying “go away.”

— And what are the most important challenges for comparison on the international level so far?

Well, it’s a very good question and I think we are still in the process of learning because we are just accumulating, and putting all the data together. So this area still needs discussion among the members. What I would say is that perhaps we have more things in common than differences, although I’m not saying I know for sure.

When we went out with the SERU-I survey we provided sample questions to our various international members –  that was 6 universities – and we thought maybe they would choose 30% of the questions as shared “core questions” - questions that are shared with our American members, and maybe 70% would be culturally driven specific questions because we know there are very strong cultural differences. In the end our members actually chose around 70% of their questions from the shared core. Of course there are some differences in language and that’s a challenge in terms of creating data that’s comparable, and we’re looking at these issues. But the theme I am trying to express is that about 70% of the questions were common. And 30% were more culturally specific customized questions.

 So I think that’s an indicator that perhaps we really do have a lot more in common. And what is in common? Perhaps the behavior is different among students, but the goals of our member institutions are similar, the seeking student engagement in the activities of teaching, research, and public service, and how our students interact and feel they are a part of a larger academic community.

— Interesting. What are the most intriguing cases of research-driven decision-making in the universities that are members of the consortium?

Yes, well, I think some of this is hard to analyze because the data is used in a broad range of areas among our members. We do a survey of our members to ask them how they are using the data, and there are groupings of activities and the data has become so common in the institutions as a reference point in many different issues that it can influence how programs are developed and structured.

At Berkeley, SERU data is used in Program Review and is an important component of that process, as I already noted. It has an influence on departments and how they structure their curriculum. And we know that engineering is one of the best users of the data at the discipline level. At a larger level we see it being used in a variety of ways. One, it’s used in the UC-System: a network of ten universities, we have common policies on things like admissions and academic advancement. The data has been used to analyze our admissions policies. How are they influencing the kinds of students we get? When we get certain kinds of students we’re taking more risk on: how do they do once they come to the university beyond graduation grades, how do they interact with others in the larger society?  So there is an area in which the data is influencing a macro-issue, like admissions.

— What do you expect from Russia, from the Higher School of Economics in terms of participation in the consortium?

We see you as an important colleague in, first, trying to understand and develop the SERU-International survey instrument. It’s an ongoing process, of course we have a core that is very stable, but we are looking for other things that will help us understand how the survey can work in an international context. So that’s one aspect. We want to work with institutions that will actually use the data. So my sense from our earlier meetings and interactions is that you will be that kind of partner. We don’t want institutions to collect data and just stick it in the corner but to integrate it into the process of academic management, decision-making.

HSE also provides another really important reference point. As I noted, we have this initial dip into various parts of the world and it’s just going to be very interesting to see how these institutions differ, and how they are similar. I think it can tell us a lot about what’s happening in higher education throughout the world. Yes, we’ve been doing these kinds of things in the US for some time and I know that you have a very sophisticated survey at your university but I think this is a tremendous window for us to really gauge what is going on globally in higher education and what kind of innovations are being pursued. So you’re providing a really important reference point in our grouping of institutions.

The other thing I wanted to note is that we will have a discussion about a potential graduate survey. We focused our efforts at the beginning on the undergraduate level but this is another important area to explore as well. And we see that collaborating with some of our international friends on a pilot SERU Graduate Student Survey will create an even stronger array of data and further reason to collaborate. Graduate education is incredibly important.  And this will be an area we hope that your university will help us think about and develop.

Prepared by Ekaterina Rylko and Igor Chirikov, specially for HSE news service