Programme and abstracts

Location: The Academic Quarter - Olav Kyrres gate 49/53, lecture room Tivoli.

(Campus map: https://www.uib.no/info/english/visitors/mainmap.html)

 

Tuesday 28 September

10.00-11.30
Preconference presentation: Introduction to bibliometrics
Dag W. Aksnes

Norwegian Institute for Studies in Innovation, Research and Education (NIFU STEP)/ University Library, University of Bergen, Norway

 

Publication and citation data are increasingly used to analyse developments in science and are also being applied as performance indicators in the context of science policy and research evaluation.  This pre-conference presentation gives an overview of basic bibliometric ghtheory, data and methods and is intended as an introduction for non-specialists. Topics which will be addressed include bibliometric databases, construction of indicators, interpretation of data, publication and citation distributions, citation and journal indicators (e.g. journal impact factor, h-index).

 

11.30-12.10
Registration at the service counter at The Academic Quarter. Welcome by this year´s host — the bibliometric group at the University Library of the University of Bergen.

12.15-12.30
Opening of the workshop by the director of the University Library, Randi E. Taxt.

 

Session 1 University rankings and institutional performance 

12.30-13.15
Key-note speak: U-Map and U-Multirank: towards transparency in (European) higher education
Frans Kaiser

The Center for Higher Education Policy Studies (CHEPS), University of Twente, the Netherlands

 

Diversity and transparency are keywords in European higher education policy. European higher education may meet the ambitions of the Modernization Agenda only if it succeeds to use the benefits of its wide institutional and program diversity. Transparency is crucial in that respect.

U-Map and U-Multirank are two instruments that intend to enhance the transparency of the diversified European Higher Education Area. U-Map generates institutional activity profiles and U-Multirank will provide information on the performance of institutions and its programmes. In the presentation both instruments will be presented. Special attention will be paid to the set-up of the U-Multirank project and the choice of indicators.

 

13.15-13.30
Beyond Rankings: Comparing Nordic Universities

Gunnar Sivertsen

Norwegian Institute for Studies in Innovation, Research and Education (NIFU STEP)

This presentation introduces a new NORIANET collaboration project in 2010-2011 that aims at developing and demonstrating bibliometric indicators at the level of the universities in the Nordic countries. As a supplement to simple international rankings of universities, it will use transparent methods and describe differences in research profiles that may explain different positions in the simple rankings. The project will serve the Nordic universities and the central research authorities in each country, as well as the organizations for Nordic collaboration at both levels, with a better understanding of bibliometric indicators and university rankings, and it will introduce more adequate and transparent performance indicators for the comparison of Nordic universities that may be updated from year to year in the future.

 

13.30-13.55
Peer-review vs. Bibliometrics. Correlation between two research assessment methods in three Swedish universities
Leif Eriksson

 Uppsala university, Sweden

 

Peer-review evaluation have generally been associated with the peer-review process in publishing but in recent years the amount of peer-review evaluations of universities and other research institutions starting with the RAE in Great Britain has become part of the research evaluation process in many countries.

In Sweden, a number of universities starting with Uppsala university in 2007 have been going through a research assessment exercise with the aim to identify research strength and potential and making strategic plans for the future. In this process bibliometrics has been used to complement or counter-balance the judgements made by the peer-review panels.

In this presentation we will examine how well the peer-review panels and the bibliometric reports correlates in three research assessment exercises, Royal Institute of Technology (KTH), Uppsala university and Lund university. It will also focus on cases where there are different outcome from the two methods and discuss possible explanations for this.

We will also discuss how the results correspond with other examples of evaluations where peer-review and bibliometrics have been used together (e.g van Raan et al, Aksnes-Taxt, Butler).

 

13.55-14.15
Coffee break

Session 2 Country performance and collaboration profiles 

14.15-14.40
The Danish wonder: What determines a nation’s excellence in research?
Jesper W. Schneider

Royal School of Library and Information Science, Aalborg, Denmark

A new report, Bibliometric Research Performance Indicators for the Nordic Countries, will be presented. The Nordic countries are compared to each other and the world average by three indicators based on scientific publishing. One of them is publication activity, which shows a significant growth in all countries, but at different rates. The per capita publication rate is also different among the Nordic countries. The second indicator is the research profiles (the relative importance of disciplinary areas of research within a country), which vary considerably among the Nordic countries. The third indicator is the citation impact of the publications from each country. All countries are cited over the World average, but Denmark stands out with a citation impact clearly above the Nordic average during the last decade. Since Denmark also performs well by the other indicators, one might ask for the reasons. This question goes beyond the observations in the report, but can be discussed at the seminar – also in terms of appropriate methods to find out.

 

14.40-15.05
International Research Cooperation in the Nordic countries

Johan Fröberg

Swedish Research Council Department of Research Policy Analysis, Sweden 

Another new report, International Research Cooperation in the Nordic Countries, will be presented. Here, international collaboration in research is analyzed on the basis of the authors’ addresses in scientific publications from the Nordic countries. More than half of these publications have authors’ addresses in more than one country. International collaboration in publications had increased significantly during the last two decades. The pattern shows close and intense collaboration in research among the Nordic countries, but increasingly in combination with non-Nordic countries. The Nordic countries have somewhat different collaboration patterns measured as the relative importance of collaborating countries in each binary relation.

15.05-15.15
Coffee break

Session 3 Bibliometrics and science policy 

15.15-15.40
Danish Research Policy 1969-2010

Peder Olesen Larsen

Marievej 10A, 2, Hellerup, Denmark

 

The analysis of Danish research policy is based on knowledge about the major actors and shareholders in public research, the reasons for and the output from public research, and the consumers of public research.

Research Policy in Denmark had a hesitating start in the 1960es. A full-fledged Danish research policy was only created with the establishment of the Ministry for Research in 1993 and the transformation of this ministry into the Ministry of Science, Technology and Innovation in 2001.

Among the main issues in Danish Policy are:

1.         The conflict between basic research and research giving immediately useful results

2.         The development of the research council structure and the conflict between the outside funding of university research and the funding of university research from basic budgets

3.         The budgeting of research within the basic budgets of universities

4.         The dependence or independence of universities and of research councils from government and parliament

5.         The distinction or lack of distinction between peer review and decision making

6.         The economic output of research

7.         Measurements of research and research output/impact. What can and what cannot be measured? To what extent can the available measurements provide information about how well the purposes of public research are fulfilled?

The analysis is based on unique statistical material covering the period from 1967 to 2006 (provided by The Danish Centre for Studies on Research and Research Policy, University of Aarhus, Denmark).

Peder Olesen Larsen (2003), Kan det betale sig? Udbyttet af forskning i den offentlige sektor [Does it pay? The economic profit from research in the public sector] Økonomi & Politik 76 (4), 47-60.

Peder Olesen Larsen (2010), Stadier på Forskningens Vej. Dansk Forskningspolitik i Går og i Dag [Stages on the Way of Research. Danish Research Policy Before and Now] Syddansk Universitetsforlag, pp. 1-371. ISBN 978 87 7674 524 0

 

15.40-16.05
Funding competition and research performance among Finnish universities
Otto Auranen, ^{1, 2} Reetta Muhonen¹ & Nina Talola ^1
1 Unit for Science, Technology and Innovation Studies, Institute for Social Research,
University of Tampere, Finland
² The Federation of Finnish Learned Societies, Finland

 

Science policy makers in Finland have for the past couple of decades aimed at improving the research performance of universities. Competition for money has been a central policy instrument to achieve this goal. Since universities are resource-dependent organizations, policy makers assume that financial incentives make universities more productive in research.

Previous studies on university research performance usually make no connection between research performance and policy environment, for example in terms of funding. These studies also tend to concentrate on making temporally cross-sectional comparisons of universities instead of longitudinal analysis of universities' own development in regards to research performance.

In this paper we scrutinize the connection between funding competition and research performance in the context of four Finnish universities from the early 1990s to the mid-2000s. We ask: has the funding competition increased in universities, what is the development of research performance of universities, and has the competition for funding enhanced the research performance? Data include statistics and documents. Preliminary results of our ongoing research indicate that funding competition has tightened in all the universities, but the relation between funding incentives and research performance is ambiguous.

16.05-16.30
University libraries, research evaluation and bibliometrics
Maria Forsman

Helsinki University Library, City Centre Campus Library/Social Sciences, University of Helsinki, Finland

 

Bibliometrics is the very specific method of library and information studies. For many years it was unpopular in this field, especially in Finland, and still there is only a handful of researchers, who are interested in and who can do bibliometric analyses. The main reasons of the heavy criticism towards bibliometrics were the quality and contents of databases used in bibliometric analyses, as well as uncritical use of the results.

Lately the attitude has changed. There are available more databases than ever on the Internet  for researchers. Also the tools for bibliometric analyses have been developed. The need for research evaluation has raised interest in bibliometrics. The difference between fields of sciences has been noticed - the research culture and ways to publish research results vary from medicine to humanities. Also the qualitative aspect has been discussed in bibliometrics.

Now we ask, what is the role of libraries and librarians in the evaluation of research results? This paper is dealing with the new tasks of libraries in supporting research. An empirical example deals with research networks in the social sciences at the University of Helsinki. The data of the bibliometric analysis is JULKI database collected by Helsinki University Library.

18.00-
Optional dinner at self cost at Ulriken 643 mountain restaurant (http://www.ulriken643.no/en/Home/) Transportation by bus and cable car. Departure from The Academic Quarter - Olav Kyrres gate 49/53 18.00.

Wednesday 29 September

Session 4 – Methodological issues 

09.00-09.25
Harmonic Partitioning of Authorship Credit Facilitates Bottom-up Bibliometric Research

Nils T. Hagen

Faculty of Biosciences and Aquaculture, Bodø Regional University, N-8049 Bodø, Norway

 

Harmonic counting has recently been validated as an ethically sound procedure for the unbiased, bottom-up partitioning of authorship credit among the coauthors of a multi-authored publication (Hagen 2010)*. Harmonic partitioning of authorship credit facilitates bottom-up bibliometric analysis by correcting for the equalizing bias generated by unwarranted top-down allocation of equal credit to all coauthors. The impact of equalizing bias can then be analyzed quantitatively by directly comparing biased and unbiased estimates of individual publication performance.

It is now clear that unwarranted equalization creates spurious results by removing all evidence of individual achievement from the source data, irrespective of whether the bias is generated by fractional or inflated allocation of equal credit.

The effects of equalizing bias arise at the most basic bibliometric level, are compounded in aggregative analyses, and will, unless corrected at source level, affect comparisons of research teams, institutions and geographical subdivisions.

Equalizing bias also has a considerable capacity to distort derived scientometric indices, rankings and reward schemes by systematically skewing the source-data in favor of secondary authors at the expense of primary authors.

In conclusion, the benefits of harmonic counting and a bottom-up approach include greater accuracy, transparency and accountability that ultimately may combine to produce better bibliometric research, new applications, and improved research policy.

09.25-09.50
On citation indicators

Peder Olesen Larsen¹ and Markus von Ins²

¹ Marievej 10A, 2, Hellerup (Denmark)

²Institute for Research Information and Quality Assurance (iFQ), Bonn (Germany)

 

Citations and citation indicators play an important role in science studies and science policy. Often they are referred to as “impact indicators”, “indicators of the quality of research” and even “excellence in research”. Despite of this importance of citation indicators for science and science policy, there is no clear methodological base and there are many open questions. How shall the citations be attributed to the authors, institutions and countries in cases of publications with several authors? What are the advantages and disadvantages of the methods chosen for the indicators? What is the influence of the skewness of the citation distribution on the indicators derived?

Following Gauffriau et al. (2007) and the mathematical theory of measures and integrals (Halmos 1950) we introduce a general concept of weighted publications and especially citation weighted publications. This leads to a methodological base for the citation indicators. The various “classical” citation indicators, for example the citation rate (cpp), the Journal Impact Factor (JIF), the percentage of publications cited at least once, the Hirsch-Index h and some of the indicators derived from these “basic” indicators are discussed. A point will be made on the construction of robust citation indicators.

M. Gauffriau, P.O. Larsen, I. Maye, A. Roulin-Perriard, M. von Ins (2007); Publication, cooperation and productivity measures in scientific research; Scientometrics 73, 175-214.

P. R. Halmos, (1950), Measure Theory, Van Nostrand Reinhold, New York.

 

09.50-10.15
Critical issues in science mapping: Delimiting fields by journals and the influence of their publication activity
Jesper W. Schneider

Royal School of Library and Information Science, Aalborg, Denmark

Journals are the preferred units when fields, disciplines or similar constructs are delimited a priori in research evaluation and mapping studies. The modus operandi in mapping studies most often resembles the approach outlined in White & McCain (1998). This presentation focuses on two intimately related methodical issues in mapping studies, publication and reference characteristics of selected journals and their authors. Like so many others, we discuss these issues in relation to the field of “information science” (IS), and we focus upon the journal Scientometrics.

10.15-10.30
Coffee break

Session 5 – Studies of disciplines

10.30-10.55
The 7 Phases of Pharmacological Research

Birger Larsen

Royal School of Library and Information Science, Copenhagen, Denmark

 

10.55-11.20
Changes in publishing behavior in engineering and the humanities. Analysis of two Finnish universities’ publishing patterns in 1997-2008
Hanna-Mari Puuska

Unit for Science, Technology and Innovation Studies (TaSTI), University of Tampere, Finland

 

As in most developed countries, the main aim of Finnish science policy is to improve the level of science internationally. Therefore, universities are encouraged to publish more in international, refereed journals of high level. At the same time, the societal impact of science is also strongly emphasized in Finnish science policy. This may lead to a more diverse publishing behavior.

International journal publishing is a predominant publishing pattern for natural and medical sciences, but in other disciplines other forms of publishing are more typical. The presentation focuses on the development of publishing activity in the fields of engineering and the humanities. The analysis explores the following publication forms: monographs, articles in edited books, articles in conference proceedings, and articles in journals. Also the shares of national publications and non-academic publications are examined. The analysis is based on data collected from the publication registers of two Finnish universities in three periods: 1997-1998, 2002-2003, and 2007-2008.

11.20-11.45
Analyses and evaluations of the academic activities in Humanities and Social Sciences in Taiwan

Chia-Hao Hsu, Yu-Ling Luo

Science & Technology Policy Research and Information Center, NARL, Taiwan

Paper published is a way to demonstrate the academic research performance. Due to the subjective judgments, the uncertainty of humanities and social sciences is higher than that of natural sciences. Thus, a larger reference scope and a verifiable bibliometric method are needed for evaluation. It ought to be clarified the output styles first then be developed the appropriate indicators based on the data gathered. This proposal is to utilize a fully integrated database about the books and papers of Taiwanese university scholars to establish the academic evaluation indicators. The contours of collaboration styles in knowledge diffusion among the scholars would also be sketched. This study could not only give guidance to evaluate academic performance in humanities and social sciences, but also provide the references to the authorities when they formulate the related rewarding and evaluation systems in the future.

11.45-12.45 Lunch break

Session 6 – Citation network and co-word analyses 

12.45-13.10
Citation Network Analysis of Research Articles in Environmental Engineering and Water Resources

Kai-Lin Chi ¹, Ping-Lun Huang ¹, Chia-Hao Hsu ¹, and Ming-Kwen Tsai ^2,3

¹ Science and Technology Policy Research and Information Center, National Applied Research Laboratories, Taiwan

² Department of Electronics Engineering, Tungnan University, Taiwan

³ Graduate Institute of Patent, National Taiwan University of Science and Technology, Taiwan

 

Environmental engineering and water resources are applications of science and engineering principles to improve our environment. To solve the complicated environmental problems, these applications involve a broader scope of scientific disciplines from basic science to applied science. Therefore, the knowledge linkage in the subfield is sophisticated. Research articles published in scientific and professional journals are regarded as major media carrying knowledge of researchers and scientists and thus are used in citation network analysis to study knowledge flows in interdisciplinary research.

This study reveals the knowledge flows within of the journals publishing papers in environmental engineering and water resources from 1998 and 2007. The networks of the citation analysis between fields show that the research in these two fields gained knowledge from 261 research fields and also disseminted knowledge to 244 research fields. These results indicate that the research findings in these two fields were also very useful in a variety of fields. The core journals those play important roles as knowledge reservoirs in their respective fields were also found. These core journals are in a better position to help researchers from other fields look for resources of related and complementary knowledge in these two fields.

13.10-13.35
Co-word analysis in multilingual thesaurus construction

Susanna Nykyri

School of Business and Economics, Department of Information Studies, Åbo Akademi

University, Finland

This paper reports how bibliometric co-word analysis was used as a qualitative method in multilingual thesaurus construction research. It was used to evaluate translatability of certain British-English social science indexing terms into Finnish language and culture, and combined with discourse analysis. The main data collection method was focused interviews and keyword searches in databases. The material used in bibliometric analysis was word associations of different thesaurus users groups (i.e. Finnish and British social scientists, indexers and thesaurus constructors), and indexing terms of some British and Finnish databases. The occurred problems were traditional - harmonization of the material, differentiating quality and practices of the databases, and the interpretation of what is not visible in the maps. The co-word method was considered useful in illustrating the semantic lexical network and in helping to define the pragmatic meaning and type of equivalence of the studied terms.

13.35-14.00
European Union Politics – a journal and its invisible college

Stina Johansson

University College of Borås

 

In my master thesis “European Union Politics – a journal and its invisible college”, I used a combination of methods, Author Co-citation Analysis (ACA), Social Network Analysis (SNA), and Multidimensional Scaling (MDS), to analyze and visualize an invisible college.

The invisible college concept is said to have as many interpretations as users. Here invisible college was interpreted as a network of researchers 1) who are active in the same field of knowledge 2) who are not bound to physical institutions 3) who’s names and publications can be derived through a channel of publication (in the case: a core journal).

The invisible college that was analyzed was derived from a channel of publication: the core journal European Union Politics. This journal can be said to represent the field of knowledge with the same name. Data was collected from the ISI Social Science Citation Index for two separate intervals of time: 2003-2004 and 2007-2008. This enabled a comparison of the earliest and the latest data available. The study showed changes in the invisible college: in terms of citation frequencies, density, and degree centrality. These were visualized through the MDS maps. The qualitative interpretation of the quantitative results suggests a shift in research trends towards the European Union Politic’s subfield “public opinion research”. This trend was then put in relation to previous meta-studies and understood in the lights of changes in the research object: The EU.

14.00-14.30
Coffee break

Session 7 – Other studies

14.30-14.55
The growth of research: The output of dissertations in eight research fields

Björn Hammarfelt¹ & Jens Peter Andersen²

¹ Department of ALM (Archives, Libraries & Museums). Uppsala University

² Medical Library, Aalborg Hospital & Royal School of Library and Information

Science

This paper studies the production of dissertations in eight research fields in the natural sciences, the social sciences and the humanities. In using doctoral dissertations it builds on De Solla Prices (1963) seminal study where PhD dissertations were used as one of several indicators of scientific growth. Data from the Proquest: Dissertations & Theses database covering the years 1950-2008 were used to depict historical trends. A decline in the growth of dissertations can be seen in all fields in the mid-eighties and several fields show only a modest growth during the whole period. Similarities and differences in growth were identified and it seems that applied research fields, like biomedical research and to some extent sociology, have a greater growth than fields like literature studies or physics. Moreover, there seems to be a correspondence between the number of new journals and the number of new researchers in a specific research field. Consequently, we propose that the output of dissertations can be used as an indicator of growth, especially in fields, like the humanities, where the counting of journals or articles is less applicable.

14.55-15.20
Analyzing macro structures from a micro perspective: The Michel Foucault example

Fredrik Åström

Lund University Libraries, Head Office, Sweden

Bibliometric analyses of research in the humanities have been considered problematic for a long time, in research impact analyses as well as mapping, where e.g. endeavors towards mapping the whole ‘research universe’ often represent the humanities as isolated and peripheral. This investigation is addressing two questions related to the impact and role of the humanities in a larger academic context. The first question is whether we can find ways of mapping the influence of humanities research on academia in general that does not rely on journal-to-journal citations in the same way many maps of research and impact indicators do? The second question is how to identify the lack of intra-field references/citations as an expression of research operating more on the basis of distinctive peer-to-peer relations, rather than a cumulatively organized research where peers cite each other as a sign of intellectual debt? By using the ‘Cited reference search’ in Web of Science (WoS), all documents citing works by the French philosopher Michel Foucault, arguably one of the most influential humanities scholar in the late 20th century, was retrieved. By analyzing WoS Subject Categories, the influence of Foucault on different fields of research was investigated; and through a citations-among-documents journal co-citation analysis, the difference between the total amount of journals citing Foucault was compared to the number of journals also citing journals within the document set.

 

15.20-15.45
Google Scholar and ISI WoS. Author metrics within Earth Sciences subjects
Susanne Mikki, University Library, University of Bergen, Norway

 

Since Google Scholar has been released in 2004, the amount of content has increased considerably, not least due to database vendors, journal publishers and scholarly societies who provide their content to this service. To which degree Google Scholar challenges the position of ISI WoS as a leading citation service is investigated by comparing author search results within Earth Sciences subjects.

In this regard, Google Scolar's recall of items indexed by ISI WoS is determined, and Spearman's footrule is used to find similarity of rankings. Furthermore, for comparing author performances, the number of publications, citations and the h-index are calculated.

 

15.45-16.00
Closing of the workshop and handing over to next year´s host