Jump to: Abstract || Introduction || Literature Review and Evaluation || History of the TULIP Program || Methodology || Results || Discussion || Appendices || Notes || Literature Cited
Abstract
Installation of the TULIP (The University Library Program) system from Elsevier Science Publishers has made electronic and paper journal comparison possible at the Engineering Library at the University of Michigan. User statistics from the TULIP system ar
e utilized to assess usage of the system, and these are compared with bound and unbound paper journal data for the same time period. Results of a survey designed to appraise access to the paper and electronic journals are also presented. Evaluation of the
results indicate that some characteristics of the three venues of the electronic version (TULIPview, MASC, and WebTULIP) have been distinctly different over the years of the project -- e.g. number of requests and journals used -- and that other character
istics have been similar -- e.g. request format and viewing/printing accesses. Lack of available data and the difficulties in comparing two noticeably different access venues hindered comparison of electronic venue use characteristics with use characteris
tics of the print version, although some roughly comparable results are presented. Other similar systems to TULIP are discussed, and needs involved in creation of such systems are also evaluated.
Introduction
The number of electronic journals and electronic versions of paper journals has increased
substantially with the increased availability and use of the Internet. Several thought papers
have been published that review the advantages and difficulties associated with publishing
journals in electronic format. (1) However, we still await a study comparing the differences in usage of electronic and paper journals. Such a study might contain important information on how the electronic journal format has been receive
d, and might also provide advice on what to include and what to be cautious of when creating information systems that house such electronic journals.
The advent of the TULIP project at the University of Michigan made the opportunity for such a comparison possible. The TULIP project is sponsored by Elsevier Science Publishers. It is a research project designed to test electronic journal dissemination an d access within a number of academic libraries (2) -- implementation at the University of Michigan began in July of 1993. Both the Engineering Library and the Science Library (3) housed the paper versions of the journals Elsevier wished to disseminate electronically through the TULIP project -- in the case of the Engineering Library, 46 Materials Science journals (see Appendix B). Details of the project will be discussed in a later section.
Because of the implementation of this project at the University of Michigan, comparison is possible between the paper versions of the journals available in the Engineering Library and the electronic versions of these journals contained in the TULIP system. To adequately make such a comparison, electronic request and print circulation/browsing journal data were acquired for the years the TULIP project was implemented. This paper will lay out the methodology involved in getting this data for comparison, the manipulation of that data, and the results obtained. A survey was also created and disseminated to Materials Science researchers to gather views on access to the TULIP materials in paper and electronic form. The results of this survey will also be presented. A literature review of the information- seeking behavior of those using print and electronic formats will be evaluated, and various electronic journal information systems will be talked about as well. The history of the TULIP project will also be sketched out. The discussion will assess the import and value of the data and the TULIP system as a whole, with suggestions for data collection within such systems.
Any statistics relating to journal titles are confidential and for Elsevier's knowledge alone. Therefore, journals are not named, but statistics are presented using designations for the journals (i.e. Journal A, B, etc.).
Literature Review and Evaluation
The information-seeking behavior of users has been well researched. In this paper, I address browsing in particular, as I feel this is an important and misunderstood aspect of
information systems, paper and electronic. I will also cover a number of useful case studies
that suggest evidence of differences in information-seeking between disciplines and
personalities. I will then discuss the importance of providing access to electronic information in a way that makes it digestible to users and hence my focus will shift towards detailing the variety of networked electronic information systems available --
gateways to electronic materials, CD-ROM LANs, World-Wide Web implementation, electronic journals. Chief among these are the full-text accessible systems, of which TULIP is one.
Strategies involved in this type of behavior are bi-modal: seeking specific information or using insights into the information problem to lead to the necessary information. Both of these strategies work in the partitioned general model above. For instance, they are both used in browsing. Knowing what one is looking for yet not knowing where to go to get it reflects semi-directed (semi-specific) browsing, while having no idea of what to look for and scanning all available materials for those that may fit is known as random (intuitive) browsing. (5)
Browsing is a searching strategy not often taken into account in indexing and abstracting information services. Many recent studies point to the fact that browsing is one of the more important methods of collecting information -- for instance, it allows serendipity to occur. Older studies seemed not to understand the importance of browsing. Many researchers in library and information science still do not understand the importance of browsing, due to inaccurate pre-defined notions of information-seeking -- i.e., users have specific information needs, and know what they want and how to get it. (6,7)
It is important to realize that browsing in a paper system (catalog, paper indexes and abstracts, current journals shelf) cannot be exactly equated with browsing in an online system. For instance, while users can easily and quickly utilize the browsing capabilities of a current journals shelf, studies have shown that users may not browse as often as they could in an online system because they do not have the time to invest. The system may be more accessible to those who have more of a specific information need because of the time problem -- in essence, more amenable to searching. (8) In my mind, this makes creation of superior browsing interfaces in electronic information systems all the more necessary -- the more facile and friendly the browsing interface, the less time the user needs to navigate it.
Browsing in information systems can be done for a variety of reasons -- random or semi- directed browsing as in paper, and also to learn the navigation of the system if users are new to it. Users generally browse in these systems by utilizing lists of terms (or journals), or by perusing a list of results (or articles) for relevant materials. Results listed by relevance to the initial query and better classification of browsable terms can offer even better navigation functions for users. (9)
One of the barriers to browsing is the complexity of the information as presented and the facility of the information system. (10) Hypertext information systems provide the most useful forums for browsing. Links within and between various files allow navigation of the system in ways that benefit serendipitous browsing. Good browsing interfaces are best achieved by designing the system so that its structure encourages browsing and it places little demand on the user's goals. Efficiently created structure allows for backwards browsing, reviewing classification and subject listings, and re-inputting a search on a subset of retrieved items. (11) Although most of the research to date has been done on OPACs, that is not incongruous with this study -- much of the content and structure of TULIP involves bibliographic citations.
From the above discussion, it is not entirely evident that different disciplines need different electronic information systems devised for them. It seems that more personal aspects of users and other factors such as gender and department tenure may be more important in information-seeking behavior than discipline. If this is the case, and certainly more study needs to be done in this area, it may not be possible to create information systems that cater to a particular group. It is important, however, not to discount the broad needs of certain disciplines -- for instance, chemists need access to chemical structures, most often in diagrammatic form. The most essential aspect of creating an information system is to first ask the users what they need before the creation of the system, so that at least discipline- specific factors can be taken into account, if any exist.
As mentioned earlier, another essential element of information systems should be the ability to browse. Browsing has been misunderstood and unimplemented for so long, but has received a reprieve from a newcomer on the scene: hypertext. Newly created electronic information systems need to take this into account, since the functionality of hypertext vastly increases the accessibility of information in a system. For instance, browsing the journal issues (e.g. July 1994) in a particular electronic journal can offer the user another method of obtaining possibly new information in a subject area when a particular keyword search does not come to mind. As well, concept linking (in hypertext) between parts of an article or different articles also provides the opportunity for serendipitous findings.
Other important factors that need to be taken into account include:
As examples of advances in these areas that have helped developers of electronic information systems, the Z39.50 protocol has increased the potential for systems to communicate without the problem of setting separate protocols for connection with data providers. The need to store large quantities of data has increased as data providers are able to provide more data to users. And the Internet, in particular the World-Wide Web, has granted users remote access to a profusion of materials not before available.
For information to move from the creators to the users, it has always had to go through intermediaries, such as catalogs or indexes. Users now often need to learn how to navigate electronic information systems as well. Since technological advances are very rapid, what is available technologically at the time of creation of an electronic info rmation system is what the users have to work with, which may make the search for information more difficult in some respects. Discussion of some of the different electronic information systems that have been created, focusing on access to journals, will highlight some of the innovations and hindrances to providing unlimited access to users, with discussion of the history of TULIP as a separate section.
More than 30 electronic journals are published via the World-Wide Web and not in paper. Advocates of electronic journals stress that the increased speed by which researchers can receive articles and, in particular, the data from researchers which is known before article publication, can aid in increasing scholarly communication. (16) However, there are drawbacks to providing electronic journal access. Chief among them is the reputation of the journal in its field -- without notice of its import, a journal will not be easily accepted. In particular, without the acceptance of those involved in tenure decisions, an electronic journal will fall far short of its goal. It is the opinion of library directors that the creation and dissemination of electronic journals should be advanced, yet academic adminstrators are more hesitant of the success of such networks and do not feel the academic community is ready for the change (17), a fact that bodes ill for those who wish to publish in electronic journals. Another large issue is the fact that many of the electronic journals are not cataloged or indexed and abstracted in major services, therefore limiting access and decreasing their value in the eyes of readers and administrators. (18)
So interestingly, in the case of electronic journals, the biggest problem for increasing usage may not be the technology, but the behavior of these readers and administrators who are not willing to accept electronic journals because there is not likely to be a corresponding change in the attitude of those involved in the tenure decisions -- traditi onal views are still in the majority. The same circular problem is evident in the lack of bibliographic control of these journals -- access is limited without bibliographic control, and unless the journals are accepted by the academic community, they are not deemed worthy of bibliographic control...therefore, access is limited.
The further step in providing access to electronic journals has been to provide access to electronic versions of paper journals. One interesting experiment has been the TORPEDO project at the Naval Research Laboratory (NRL) in conjunction with the American Physical Society (APS). Physical review letters and Physical review E are being shipped to NRL by APS hot off the press. They are scanned and converted to ASCII text using Optical Character Recognition (OCR), uploaded along with bibliographic citations sent by APS via email, and accessed by a test audience through the World-Wide Web. The Web imaging interface for display and printing capability is through a proprietary system known as EFS. One of the problems with EFS is that it does not support the new SGML (19) standard for structured documents, which is being offered by APS in exchange for the paper documents. (20) This highlights the difficulty of decisions that are made in the initial stages for which applications should be used in a project -- knowing what technologies, and most importantly what protocols and standards, will be available and viable in the future is not always po ssible.
TORPEDO is searchable via direct keyword searching and content searching using a full-text retrieval mechanism known as Adaptive Pattern Recognition Processing (APRP). Specifically interesting to this project is the browsing mechanism. Searchers may browse hierarchically through a journal menu structure (journal, issue, article...), and may also access the tables of contents of each issue of a journal volume. (21) This structure mimics the browsing behavior of researchers on current issues shelves in a library. Since the TORPEDO project is still in progress, it is not yet known what user reaction to this type of information-seeking infrastructure is.
Another project that has been in existence nearly ten years is the Chemistry Online Retrieval Experiment (CORE) sponsored by the American Chemical Society (ACS), OCLC, Chemical Abstracts Service and Bellcore, and housed at Cornell University's Albert R. Mann Library. Core journals in the field of chemistry are sent to Mann Library to be scanned for image page production. At the same time, ASCII representations from the ACS-compiled database are gathered and converted into SGML coded documents. (22) The project researchers felt the encoding was important to test enhanced functionality of the system. (23) These researchers have found that creating the interface for the users was not their biggest problem, but that populating their database was. Since chemical journals have a great number of graphics (chemical diagrams, etc.), it was important to provide access to these graphics in an acceptable manner. They had to be extracted automatically from the ASCII text and set up to be viewed correctly -- a very complicated process. (24)
Part of the focus of this project was researching the preference for reading on paper rather than on screen. Bitmapped images were continually made available during the project partly due to the ease of printing, and SGML coded documents have been highly touted by the project researchers as they make data presentation for a variety of display mechanisms possible. (25) Searching and browsing behavior was evaluated in experiments designed by chemistry professors to test paper and electronic journal differences in this realm. Results included:
Serendipity increased through browsing, indicating the need for reinforcing this type of information-seeking behavior in the creation of future information systems. Of similar interest is the result that pictures were looked at before text -- this may be an artifact of the chemistry field, but it is an important factor of electronic information systems not to be overlooked.
Materials science journals were chosen since researchers in this field seemed to be comfortable with computers and would have enough of them to make the results significant, and it was hoped generalizable across disciplines. An electronic journal's scanned material would be provided free of charge to those project universities that already subscribed to the paper version. This material was made available to the u niversities by scanning the original pages and compressing the resulting images. The images were then OCRed to create a flat ASCII text file for searching purposes. The images, ASCII files, and bibliographic citation information were sent to Engineering Information, Inc. for distribution to the various university sites via File Transfer Protocol. Unique sets were created for each university corresponding with the journals already subscribed to by those universities. (29)
TULIPview is a unique application that provides access from Unix machines to citations with abstracts and the full text of articles. A user can display and print images from within the application to his screen and selected printers around campus. In addition, with TULIPview, he can set up a profile for himself of the information he would be most likely to use, and citations of new articles that have recently been uploaded into the system can be sent to him via electronic mail. This is known as the Selective Dissemination of Information (SDI) service.
Access to TULIP is also made possible as an MDAS (Multiple Database Access System) file via the MIRLYN library catalog interface. This file, called MASC for Materials Science, provides access to citations with abstracts and the full text of articles. Howe ver, it is not possible to display page images, although printing of the full text of articles is available, as with TULIPview.
TULIPview is essentially a searching tool -- a user can enter a search keyword, limit it by date and journal type, and retrieve a list of articles starting with the most recent listed first. However, it is not possible to retrieve a table of contents for an issue of a journal. The best a user can do is limit the search by date (i.e. 7/1/94 to 7/31/94). This restricts the browsing capabilities of such a system, and is only roughly analogous to browsing the unbound journal shelf. MASC is similar in that it is not possible to view any of the full text of the material provided, so although a user can call up a list of a particular journal's articles (using the .jt. search), he cannot view them in issue-by-issue format .
WebTULIP was initiated in 1995 at the same time as 19 additional journal titles were added to the initial subscription list of 27 titles at the Engineering Library (see Appendix B). WebTULIP became the first real browsing interface for TULIP. It provides access to citations with abstracts and the full text of articl es, as with the previous access venues, and it also allows for much easier searching by journal and accessibility of rough tables of contents. A user can peruse a list of the journal titles, click on one and receive a list of the issues for that journal. From there he can click on a particular issue and the system will return a list of the articles in that issue, along with links to citation/abstract and page images of that article.
For WebTULIP it was necessary to provide a special print application for users. This application allows printing of the TIFF image at a high enough resolution for reading purposes. It is possible to print the GIF image off the World-Wide Web browser application by using its print option, but the quality of the printed images is not sufficient.
In January 1995, Elsevier conducted 3 preliminary interviews with faculty. Thirty-four full interviews with faculty and graduate students were held in February and March of 1995. Nine of these interviews were with graduate students during a focus group se ssion moderated by Glickman Research Associates, a firm under contract to Elsevier for evaluation of TULIP objectives. Ten were one-on-one, in-depth interviews for approximately one hour in the offices of faculty and graduate students. Fifteen were teleph one interviews with "aware non-users" (i.e., people who had heard of TULIP but never tried it or used it only very infrequently). Several of the major findings from these studies are:
TULIPview and WebTULIP data were obtained remotely via the Web -- the data was compiled and compressed and a link made available to this data. Once uncompressed, the data was initially viewed via pico in the Unix system. The data as obtained consisted of many elements listed on the same line with spaces in between the elements. One line of data recorded one search or browse attempt in the TULIP system, culminating in a citation or image file display, or a print request. This data contained these elements:
Perl scripts were created to break the data into easily manipulated portions. Specific elements were extracted from the lines as presented and imported into a new file. So, for example, one of the new files would have only the elements corresponding to the ISSN of the journal and the user department. I later found that these were not necessary when working in Microsoft Excel, as I could correlate columns of elements within the complete file (all lines of data).
Six of the elements were used: date/time of request, ISSN, manifestation of request, user department, user status, and delivery method. (34) Histograms were calculated for the frequency of requested journal titles within a month for both TULIPview and WebTULIP. This frequency was graphed opposite each journal title. (chart1, chart2) Each journal title was given a number for ease of use within Excel (histograms could only be calculated using numbers, and the ISSNs were unwieldy in calculation and aesthetically unpleasing in chart presentation). This information was also graphed for the years 1993 through 1995 for TULIPview. (chart3) Comparisons between TULIPview and WebTULIP were made for November of 1995 (chart29), and all of 1995 (chart30).
User departments, user status, delivery method, and manifestation of the request were tracked using the same methods for each element. Data for these elements were graphed by month. For instance, user department data were graphed opposite the frequency of requests and months. Seven user departments were classified by the University of Michigan TULIP team:
User status was separated into:
Delivery method was made available to the user in three ways:
And manifestation of data was made available to the user in one of two ways:
TULIPview data for 1993-1995 was graphed, as was 1995 data alone. The 1995 TULIPview data can be compared with the 1995 WebTULIP data, graphed in the same manner. (chart5:chart12, chart7:chart13, chart9:chart14, chart11:chart15)
MASC data was compared with the TULIPview and WebTULIP data by year. The only available usable MASC data were monthly counts of screen displays. This is roughly analogous to the monthly number of requests within the TULIPview and WebTULIP systems. Data for 1993-1995 were graphed for the MASC and TULIPview data by frequency of requests/displays and month. (chart16) 1995 data was graphed for MASC, TULIPview and WebTULIP data in the same way. (chart17)
Bound journal circulation data was graphed for 1990-1995 by journal title and frequency of circulations of bound volumes. (chart18) This data was also graphed by month and frequency for each year. (chart19) For ease of viewing, it is also graphed for only the years 1990, 1993, and 1995. (chart31)
Unbound browse data was graphed in a similar manner to chart18, except that it was split into two parts -- 1990-1992 and 1993-1995 -- for ease of comparison with TULIPview and WebTULIP. (chart20, chart21) Chart21 could be compared with chart3 (TULIPview data by title), because browsing unbound journal shelves in the library can be seen as roughly comparable with navigating the TULIPview system. A case can be made that this is not true due to the lack of browse mode element data for TULIPview, as spoken of above. It is in fact more effective to compare the shapes of the graphs (both venues' requests dipped in November...) rather than the total number of requests (there seem to be more requests for TULIPview than unbound journal browses...). Another reason for comparing the shapes and not the numbers is that unbound journal tallys are not accurate numbers. Although we ask at the Engineering Library that patrons not re-shelve the journals, many of them still do. As well, re-shelving during high workload times and end of terms tends to be erratic.
Unbound browse data, TULIPview and WebTULIP data were compared for 1995 by journal title and frequency of browses/requests. (chart22) Unbound data were not graphed by month and frequency, because they were not available split into months.
Researchers were asked which Elsevier titles they used, as well as questions on how they used the paper journals within the library and the electronic services via TULIP. These questions were composed in Likert Scale format. As well, researchers were asked whether some circumstances of either of the access venues made it easier or harder to find the material they were looking for. Final questions were on age, gender, status within the University, and the researcher's department. A response box was also provided for general comments, which a majority of the respondents utilized. (see Appendix A)
Thirty-three researchers responded to the survey using either the Web or email -- the first request contained a URL for the survey on the World-Wide Web, and the second contained the text of the survey in the body of the email message. Journal title usage responses were graphed as frequency of responses versus specific journal title. (chart23) The Likert Scale questions were summed (e.g. 4 responses for question #2 in Likert Scale category 1, 6 responses for question #2 in L ikert Scale category 2, etc.) and Chi-Square statistical tests were performed on each question to determine if there was significant deviation from expected results.
Since the majority of the respondents did not answer every question on the survey, summation of all the questions for each respondent would not provide a very truthful or useful statistic. However, averaging all the respondents' sums from a particular department (e.g. MSE, NAME, etc.) might yield a rough possible correlation across departments. This was performed for department and for status within the University.
In addition, it is important to note that use of certain journals is not consistent over the months. In November 1993, the most used journal was Journal 29 (70 requests), in November 1994 it was Journal 28 (113 requests), and in November 1995 it was Journal 1(120 requests) (chart1, chart24, chart25) Over the years, the number of journals used has increased (as was indicated above for months as well), and some specific journals have stayed in consistent use while increasing in use as well. (chart3) For instance, Journal 28 was highly used in 1994 and more so in 1995 (588 and 718 requests). Journal 42 was the most used journal in 1993 (195 requests), slightly less used in 1994 (123 requests) and the most used journal in 1995 (790 requests). Other journals have not been used at all or very little, and that has been consistent as well across the years. Journal 4 has not been used at all in the three years, and Journal 9 was used fewer than a dozen times in 1993 (7 requests), a bit more often in 1994 (29 requests), and not at all in 1995.
It is possible that the reason for the number of journals used increasing over the years, and in particular in 1995 (chart3), is due to the addition of new Elsevier journal titles in the beginning of 1995. (see Appendix B) For instance, both Journal 1 and 20 were added in 1995, and both were significantly used in that year (389 and 110 requests). It is also possible that the increase in number of requests for journals in 1995 was due to the introduction of WebTULIP. It could be that the installation and promotion of the new access venue made re-awareness of TULIPview possible. Total number of requests in TULIPview decreased slowly over the latter months of 1995, while WebTULIP requests increased, so although specific title use increased in TULIPview in 1995, total number of requests declined. (chart30) Re-awareness of TULIPview may have occurred, but WebTULIP seems to have been used more often.
TULIPview format data -- whether users of the system accessed citations and/or images -- is most easily looked at by month and frequency of requests. (chart10) It is obvious that both citation and image format requests increased starting in late 1994 and then decreased in late 1995, corresponding with total number of requests. (chart16) Image format was used more often than citation format, indicating that the ability to view/print images is an important function of the TULIP system, as is borne out by the views of those interviewed by Elsevier and Glickman.
An interesting anomaly of the format data is in the huge number of citation requests for September 1993. It is possible to correlate this with the correspondingly large number of electronic mail delivery (SDI) requests for that same month (chart8), and tentatively conclude that data for the SDI service for users of the TULIP system was either inadequately kept, or SDI was not used after that month.The former seems more likely, since users indicated when interviewed by Elsevier and Glickman that they had either never received information after starting the service for themselves or had received far too much. (35)
Delivery of TULIPview documents is brought about by electronic mail (for SDI), screen display and personal printout. Disregarding the electronic mail method spoken of above, it is clear from looking at the data that screen display is a much more requested method than that of personal printout. (chart8) This could be because users more often viewed the material onscreen and then printed out only those that seemed most useful. Maybe its the case that users would have liked to p rint out more than those that were printed, but found the printing speed rather slow. From my survey, one of the questions was whether the speed of printing made getting what they wanted easier or harder. A Chi-square test on the answers to this question indicate that it is inconclusive that there is a relationship between printing speed and whether this made getting what users wanted harder or easier. (Chi2=6.8, df=4, alpha=.05) However, in the interviews by Elsevier and Glickman, users did sp ecify that printing was too slow. (36)
Department and university status were consistent in the data available and what this data showed. The data indicate that most of the users of TULIPview are either Engineering faculty members or Rackham graduate students, however the data seems very incomplete. (chart4, chart6) There were many places in the data in which status and department elements were not identified. This seems to be due to the fact that from a public TULIPview station, such as the one that was located in the Engineering Library, the user does not log into the TULIP system with his login ID, from which such information is taken. From a Unix terminal or Unix prompt, personal information can be gleaned from a user's login to the Unix system before the start of his TULIPview session, but there are numerous ways that this can go awry as well. It is thought that many of the users at the public station were graduate students, so a great deal of their data may be lost. (37)
One very interesting portion of the data is the usage of the TULIPview system by researchers from Ford Motor Company. The system was made accessible to them starting in November of 1994 (this was a special case -- they did not receive access to any other University of Michigan owned electronic information systems), and it is obvious they began using it heavily from then until aroun d September of 1995. (chart4) Ford researchers were classified as Other in university status, and this corresponds with the status data as well. (chart6) It seems that making such a system available in a setting other than an academic setting produces possibly very interesting results about the differences between, for instance, industrial and academic information gathering.
Using the same month (November) for journal/requests comparisons, it was necessary to normalize the Y (frequency of requests) scale between the two venues' graphs to do adequate comparisons between the two. It is obvious that WebTULIP received more hits per journal than did TULIPview and that the number of total hits is much greater as well. (chart2, chart26) This might be attributed to the fact that it is a more accessible system from remote sites and has a better browsing interface than TULIPview. It could also be due to the addit ion of the 19 more Elsevier titles at the beginning of 1995.
WebTULIP format data shows that requests for images, and much less so for citations, increased throughout 1995. The highest number of total requests was similar (869 requests for TULIPview, 804 requests for WebTULIP), but at different times of the year -- for WebTULIP in October 1995, for TULIPview in March 1995. (chart11, chart15) This could be explained by assuming that the implementation of WebTULIP caused an increase in its use while causing a decrease in the use of TULIPview, as is borne out by a look at the total number of hits per month. (chart30)
There are similarities for WebTULIP delivery method data. In this case, screen displays exceeded personal printouts, as they did for TULIPview. While screen display went steadily down for TULIPview, it went up more erratically for WebTULIP. (chart9, chart14)
University status and department data are also possibly incomplete since sign-on to WebTULIP offered the option of choosing not to complete this information before the start of a session. However, it is evident that Engineering graduate students request the most citations/images (chart12, chart13) -- but, this correlation is puzzling. Engineering was associated with faculty members for TULIPview, while Rackham (the Graduate School) was associated with graduate students. This correlation seems to be opposite for WebTULIP. It may be that data for WebTULIP was collected in a different manner, including how the collectors distinguished between different levels of status and department. It is true that sign-on for WebTULIP was voluntary and input by the user himself, therefore mistakes could have been made by the user.
MASC use follows an academic calendar in that it is used less often from the end of Winter term (April) through the summer months and climbs again at the start of the Fall term. It also seems to be used less often in December and January. Use stayed high in these latter months in 1993, for which a reasonable explanation is that it was new and hence publicity and novelty made it more heavily used. When MASC data is plotted opposite WebTULIP and TULIPview data for 1995, it is interesting to note that the use of all three systems during the summer months was close to the same. (chart17) This trend is important for those who plan installments and updates of electronic data repositories -- in academia, the summer months are the months to perform this task.
From the graphing of the bound journals by journal title, it becomes clear that as with the other venues, certain journals are used more often than others. (chart18) More importantly, some are not used at all.
Visual inspection of the data by month shows a tendency towards an increase in use over the years. (chart19) Use dips at the end of the Winter term (April) and at the end of the Summer term (August) as it does with the other venues. Use of the 46 titles during 1990 and 1991 hovered around the 20 volume circulation mark. Use during 1993 and 1994 was around 45. Use during 1995 is more erratic in that i t started off low, but ended with the highest circulation of all the years. This could be due to two possible explanations: use went up for bound journals because of the implementation of the TULIP system and hence increased awareness of Elsevier journals , or use went up because people became more aware of the Elsevier journals because of the TULIP system but used them in paper because the articles were difficult to read on screen and/or the printed article image resolution was bad.
If a comparison is made between unbound journal, TULIPview and WebTULIP usage, the one possible conclusion to draw, since once again the numbers are unreliable, is that unbound journal and WebTULIP venues drew more specific journal use across the board than did TULIPview. (chart22) In 1995, 37 journals were used by TULIPview users, and 43 by both unbound journal and WebTULIP users. TULIPview users seem to be a bit more specific about their choice of journals, and when chosen use those journals a fair amount.
Those journals used most often for each venue are:
Clearly, both Journal 1 and Journal 28 vie for first place as the most-used journal among venues. This is important for considerations of single-use or blanket use electronic journal policies, as will be spoken of in the discussion.
Chi-square tests were performed for each of the Likert scale questions asked on the survey. The obtained results strongly indicate that this is a transition period for electronic information systems -- answers to the questions ranged widely, not showing s trong use of either the paper or the electronic journals. The most noticeable results came from the questions on:
Interface via netscape is great...There is no consensus on printing in TULIP, although copying articles out of the paper journals is regularly done. This result correlates with the TULIPview printouts and bound circulated journals comparison. (chart27) TULIPview printouts are erratic, although increasing in the later years of the project, while bound journals circulate at a rather regular rate. Although this is not a completely parallel situation (bound journal circulation vs. copying from journals), it gives an indication of the regularity of action in the print medium. Two comments from respondents give reasons why this may be so:Web TULIP is really great. It's a perfect use of Web resources.
Web TULIP is so valuable that it needs to be expanded to the premier journals in materials science, not just the second-tier journals to which it is limited.
The ability to check out journals from the library is important because the copying machines in the library are not of sufficient quality.Users often or regularly went to the journal of their choice immediately in the print format, while there was no consensus on this in the electronic format. This could be due to unfamiliarity with the TULIP system, as well as to the unavailability of a good browsing format.I generally copy articles for use later, but for large jobs, it is more convenient, and far cheaper, to use the department xerox machine upstairs.
Surprisingly, the answers to the factors of printing speed, article image quality, and downloading of the local print application were other than expected. Interviews by the Elsevier/Glickman team indicated that printing speeds and image quality were a big problem in their use of TULIP. Some indicated that they had stopped using TULIP for these very reasons. (38) However, the answers to my survey were inconclusive in that some people indicated that these factors made their search more difficult, while an almost equal number indicated that it made it easier. It is probable that respondents did not fully understand the questions, and may have gotten confused when answering the Likert Scale questions. There were many comments to the survey that indicated that the previous interview results are more accurate:
At this time, I trust we are keeping paper journals as well as electronic journals. Until image quality improves and processing speeds increase, we need both.Many of the comments provided by respondents revolved around specific journals to be added to the database -- these were invariably non-Elsevier titles and so this cannot be done in TULIP, but it shows the demand for electronic journals to be made available online.Interface via netscape is great -- lots of printing bugs, though (many crashes), also too many through Unix TULIP [TULIPview].
Printing is not up to photograph standards.
It would be nice to be able to either grab the articles as a postscript file or to use the printer to print to a postscript file.
I have to use a convoluted way to print from a remote workstation to a local printer, which I can do easily once I have a PS file. But with the printer app, I need to go to a specific machine to run it to get it to print to a local printer.
Curse the speed and reliability of the print engine, Print Local.
Other respondents indicated that TULIP journals are not that useful to them, as did many of the users asked about this in the Elsevier/Glickman interviews. Comments from my survey:
This is a great service. Unfortunately, I'm not interested in many of the journals.And the most important comment that seems to sum up the economic debate over electronic journal dissemination services:I did not know that Polymer was available on TULIP. Other than that, most of the journals that I use most are not there.
None of these journals are on my regular reading list in materials science.
I am willing to increase my use of electronic media, when it saves me time or money. That's probably the bottom line.
Discussion
Limitations of this study were numerous. One of the biggest limitations was that data
were not accessible for all electronic TULIP venues, most notably MASC data and SDI
data for TULIPview. MASC print request data could have been correlated with bound
journal circulation data more closely than with other data obtained. Both TULIPview and WebTULIP garnered a much larger number of requests than MASC requests -- MASC printout requests correlated with the same data from these two venues could be highly
enlightening.
The capture of SDI data could have provided important information on how well-received such a service could be among researchers. Consequently, another limitation is in the results that were obtained. Since information-seeking in paper and electronic journals is not necessarily comparable, any correlations made between the different formats must be taken with that caveat in mind.
The largest loss is the lack of data for WebTULIP. Since this venue has only been operational for a year, any conclusions reached from the data cannot be fairly generalized. More time and hence data on this venue would have been welcome, as data regarding the functionality of hypertext in electronic journal dissemination services are m uch needed. For this purpose, Web logfiles should be taken that track a user across the system, indicating where they have come from, which pages (i.e. articles) they choose to view, how they navigate the system, and various other characteristics.
Of the TULIP project data results that were obtained, the most interesting are:
Introduction of WebTULIP and new titles at the beginning of 1995 may have increased the use of TULIPview. In a sense, this provides support for the need for continued publicity of a system of this sort -- new portions of the system were implemented, which generated the need for publicity to bring them to the attention of users. Although initial publicity was undertaken for the TULIP project, many users had never heard of it in 1994. (39)
Although journal use rose, total use declined in TULIPview, while WebTULIP use increased slightly. This is most likely because the Web venue is a more functional venue for users, specifically in its ability to provide browsing capabilities. It is possible that this is so since both unbound journal usage and WebTULIP usage (browsing capable venues) were greater than TULIPview use. However, the data I had to work with did not allow me to conclusively demonstrate this -- logfiles of each user's navigation th rough the WebTULIP system might have been able to do so. In addition, when more data is available for WebTULIP, conclusions might be more reliable.
The fact that images were viewed more often than citations allows for speculation that full- text systems are well-received. It would be even more interesting if it were possible to determine how important displaying the image is versus just obtaining the printout of the image -- MASC printout data would have been key here. If number of printouts was low in MASC, as they were in the other venues, not much conclusive evidence could be found. If they were higher than in the other venues, this may indicate that screen displa y is important for determining choice of article printout -- without display, users must print out all interesting articles based only on bibliographic information. Also, with logfiles to track use of specific journals and the navigation patterns of brows ing users, much more information could have been gleaned about the outcome of the MASC venue.
Of the survey results, the most interesting are:
The survey results tend to reinforce the TULIP project data results, however it is important to note that there is conflict between user comments and their answers to the Likert Scale questions. It is very possible that the questions asked in the survey were too broad or too complicated for the users to understand. Likert Scale questions can also be a hindrance to getting good results, as users have to continually check back to the scale key to make sure they are answering correctly. This can cause frustration that leads to unanswered questions or incorrect answers. After examination of the TULIP project results, it became obvious to me that some of the survey questions would have been more useful split into venue-specific questions. It may then have been possible, for instance, to ask how printing in TULIPview versus printing in MASC versus printing in WebTULIP affected the users. Or whether users browsed more often in one or more of the three venues.
The answers to all printing questions were inconclusive -- an interesting result in itself. Since users ranged from those that never printed to those that always printed, it could be possible that printing may not be as important a factor as assumed in full-text systems, especially since the TULIP project data shows that users viewed images more often than printing them. The CORE project came up with an interesting result that may corroborate this -- that users absorb content on the screen in a similar way as they do on paper. (40) In my mind, the high incidence of viewing rather than printing may be due to the browsing that many users engage in when they are faced with a new article. Reading abstracts, skimming text, and checking references are browsing mechanisms used when first encountering new material. It is highly possible that these same mechanisms are brought to bear in both paper and electronic environments. If it were possible to determine how many copies users make from browsing unbound journal issues (more difficult to calculate if done outside the library) versus how many printouts they make when they are browsing an electronic system, such conclusive evidence might shed more light on the printing issue.
It is heartening that users found the WebTULIP interface a good interface. It is my hope that this was due to the increased browsing functionality, but once again the survey was not specific enough to capture this information. Further studies specifically focused on WebTULIP would be in the best interests of the proponents and creators of electronic information systems.
Specific journal usage brings up the issue of blanket access to a publisher's range of titles versus pay-per-use access. What this study has shown is that a number of journal titles have been used consistently across venues and over the years of this project. This result is an indication that pay-per-use access may be more in the best interests of a library. It must also be taken into account that the users have indicated that the journals in the Elsevier collection are not necessarily core journals in the field of Materials Science. Results from the CORE journal project, which used the core journals published by the ACS in the field of chemistry, will be very enlightening. If use was consistent across all journal titles, choosing only core journals of a publisher may be a good means of providing access for users in electronic venues. As well, this issue is important for collection management purposes -- managers are beginning to have to choose between paper and/or electronic versions of journals within limited budgets. It is possible to calculate how much it costs to hold one bound volume in a library. This study has not provided any new material toward the knowledge of how much it costs to hold one electronic volume, since the TULIP materials were provided free of charge to the project universities. It is important that such calculations be made. The University of Michigan will be receiving all 1100 of the Elsevier journals in the Fall of 1996, and a variety of economic models will be tested for this very purpose.
In conclusion, I think it is important to review the needs for the creation of an information system. The basics include:
Creating an interface must include:
In the case of TULIP at the University of Michigan, most of these criteria were met. However, several of the criteria were not fully met, namely establishing user need, conducting training, and collecting data. For this project, the establishment of user need was not possible, as it was a research project inititated through the publisher, not the library. While there was training and publicity, many primary users had not heard of the system in its second year. Problems of collection of data were enumerated at the beginning of this section.
From my evaluation of this study, other criteria have also been indicated as important to take into account. Specific fields of study and individual needs should be taken into account whenever possible in the creation of an information system. This ties i n with the establishment of user need. In addition, the need for browsing-specific interface design is apparent, as has been discussed throughout this paper. TULIPview is a powerful searching tool and WebTULIP is a powerful browsing tool -- people seem to prefer WebTULIP which supports the need for browsable information systems. Additionally, taking a cue from the CORE project, scanning paper products is not the wave of the future since this material is in the large majority created initially in electroni c form. The currently preferred format for dissemination and access of documents is in SGML which offers greater freedom of choice for those providing the access -- for instance, printing is made easier and faster with SGML encoded documents than with sca nned images. SGML encoded documents for TULIP should be made available to the University of Michigan by Elsevier in 1997.
TULIP has been a learning experience for all involved -- publishers, librarians, and users alike. Knowledge of how to implement and effectively use systems like this one in the future has been part of this experience, and will continue to be at the Univer sity of Michigan with the implementation of the full set of Elsevier journals. Having the opportunity to be a part of the pilot TULIP project, and the full implementation that follows, is highly fortuitous for the University. Follow-up to this study shoul d be undertaken with more and better data, which may provide more substantive conclusions on the use of paper and electronic journals.
