Analysis of Tweets Mentioning Works from an Institutional Repository

: Altmetrics derived from Twitter have potential benefits for institutional repository (IR) stakeholders (faculty, students, administrators, and academic libraries) when metrics aggregators (Altmetric, Plum Analytics) are integrated with IRs. There is limited research on tweets mentioning works in IRs and how the results impact IR stakeholders, specifically libraries. In order to address this gap in the literature, the author conducted a content analysis of tweets tracked by a metrics aggregator (Plum X Metrics) in a Digital Commons IR. The study found that the majority of tweets were neutral in attitude, intended for a general audience, included no hashtags, and were written by users unaffiliated with the works. The results are similar to findings from other studies, including low numbers of tweeted works, high numbers of tweets neutral in attitude, and evidence of self-tweets. The discussion addresses these results in relation to the value of tweets and suggested improvements to Twitter metrics based on IR stakeholders’ needs.


Introduction
Academic libraries have the capability to produce complex representations of higher education institutions' research impact by integrating institutional repositories (IRs) with metrics aggregators (e.g.Altmetric and Plum Analytics).Since the purpose of IRs is to collect the institutions' scholarly output, stakeholders across campus can benefit from these metrics including: faculty and students authoring content, administrators monitoring the institutions' research impact, and academic libraries managing IRs.Altmetrics (non-traditional metrics from the social web) particularly align well with the goals of IRs due to their immediate accumulation after online publication and representation of a broad audience (Holmberg, Haustein, & Beucke, 2015).These characteristics match IRs' promotion of open access publishing, specifically in terms of grey literature (e.g.presentation slides, postprints, student research) that may not receive traditional metrics (e.g.citations) (Bernal, 2013;Holmberg et al., 2015;Konkiel & Scherer, 2013).Altmetrics have been noted to indicate many attributes, including online impact, reach, visibility, attention, interest, distribution, promotion, and/or consumption of institutions' scholarly outputs (Barnes, 2015;Eysenbach, 2011;Holmberg et al., 2015;Ortega, 2016).IR stakeholders potentially can use altmetrics to showcase achievements, evaluate research, and discover scholarly outputs and/or other researchers (National Information Standards Organizations [NISO], 2016).
Literature on altmetrics has steadily been on the rise since the term was coined in 2010 (Erdt, Nagarajan, Sin, & Theng, 2016).In comparison to bibliometrics, research is in its early stages and many topics have not been explored in-depth.Specifically, research has not focused on altmetrics provided by metrics aggregators integrated with IRs.Research examining altmetrics within IRs should be targeted since it can help stakeholders understand the meaning of altmetrics available through their IRs.To address these gaps in the literature, the author examined altmetrics from Twitter, a free networking microblogging service, where users tweet information in short bursts of 140 characters (recently expanded to 280 characters) to their followers.The primary metrics aggregators, Altmetrics and Plum Analytics, offer quantitative and qualitative data from Twitter.Plum Analytics (2018) provides the number of tweets and retweets (i.e.reposted tweet).Altmetric (2017b) provides the number of tweets, retweets, and quoted tweets.Altmetric (2017a) also details the number of Twitter users who tweeted about the work, including their demographic information (geographical location and user category) and number of followers.Both provide the content of the tweets.The author evaluated tweets collected from PlumX Metrics (a product of Plum Analytics) that mention works from a Digital Commons IR.The small scale research study was based on the following questions: 1. What are the attitudes (neutral, positive, negative) of the tweets?2. Are the tweets' authors affiliated with the works?3. Are the tweets intended for a general or specific audience?
4. What hashtags are in the tweets?
The author used content analysis to address the questions.The results produced a comprehensive overview of the tweets.While these results cannot be considered universal, they spark a discussion on the value of tweets and suggested improvements to Twitter metrics based on stakeholders' needs.

Literature Review
Relevant literature to the current study has either focused on sentiment and/or content analyses of tweets mentioning scholarly works or the relationship between altmetrics and institutional repositories (IRs).Due to the topics rarely overlapping, gaps in the literature exist related to altmetrics and IRs, specifically tweets that mention scholarship available in IRs.

Sentiment and Content Analyses of Tweets
In an analysis on the correlation between tweets and citations, Eysenbach (2011) remarked on the need for sentiment analysis of tweets mentioning journal articles.He observed that the "vast majority of tweets [in his study] simply contained variants of the article title or the key conclusion, and rarely contained explicit positive sentiments…or-even less common-negative sentiments" (p.16).
Several years after this statement, researchers conducted sentiment and content analyses on tweets tracked by Plum Analytics, Altmetric, and Twitter queries that mentioned research papers from established databases (Web of Science) and publishers (e.g.JSTOR, Science Direct, Wiley) (Friedrich, Bowman, & Haustein, 2015;Friedrich, Bowman, Stock, & Haustein, 2015;Haustein & Costas, 2015;Liu & Fang, 2017;Maleki, 2014;Ortega, 2016;Thelwall, Tsou, Weingart, Holmberg, & Haustein, 2013;Tsou, Bowman, Ghazinejad, & Sugimoto, 2015;Vainio & Holmberg, 2017).The following summary of these studies' results do not include tweets mentioning other types of works available in IRs such as grey literature.The current study addresses this gap in the literature by using works from an IR that represent both traditional publications (faculty publications and journals collections) as well as grey literature (conferences and student research collections).
Content analyses evaluate tweets through coding in order to produce quantitative data from qualitative data.Liu and Fang (2017) analyzed the content of tweets in order to discover analysis factors beyond count and sentiment.Their sample of tweets contained four types of attitudes: positive, neutral, somewhat supportive, and negative (Liu and Fang, 2017).Their content analysis differed from previous sentiment analyses because they examined tweets that expressed opinions not in relation to the research paper and coded expansion tweets (e.g.conjecture, humorous response, appeals for action, etc.) as positive rather than neutral (Liu and Fang, 2017).
Although percentages for Twitter users holding PhDs may not be in the majority, Tsou et al. (2015) argued that they are higher than those who hold a Ph.D. in the general public.Regardless of their status in academia, individuals engaged in tweets more than organizations (Haustein & Costas, 2015;Tsou et al., 2015).Several studies noted individuals tweeting about their own works, which is considered an indicator of promotion rather than impact (Liu & Fang, 2017;Maleki, 2014;Ortega, 2016;Thelwall et al., 2013).
The results of the content and sentiment analyses informed discussions on how to improve Twitter metrics (Liu & Fang, 2017;Thelwall et al., 2013).Liu and Fang (2017) favored a weight system in which positive tweets would be weighted more than neutral tweets.They proposed omitting tweets featuring an affiliated author, opinions unrelated to the paper, or somewhat supportive and negative attitudes (Liu & Fang, 2017).In opposition, Thelwall et al. (2013) argued that negative citations do not need to be separated due to their rarity.These suggestions are primarily based on authors' application of Twitter metrics, which does not address the variety of perspectives from altmetrics' stakeholders.
The current study addresses this concern by considering the viewpoints of all IR stakeholders (faculty, students, administrators, and academic libraries).

Altmetrics and Institutional Repositories
Scholars have discussed the capabilities of integrating altmetrics with institutional repositories (Bernal, 2013;Callicott, Scherer, & Wesolek, 2015;Konkiel & Scherer, 2013;Madjarevic, 2015;Melero, 2015;Roemer & Borchardt, 2015a;Roemer & Borchardt, 2015b;Roemer & Borchardt, 2015c;Tzoc, 2016;Wesolek, Scherer, Callicott, & Marsh, 2015).The following outlines the potential benefits and use cases for IR stakeholders when altmetrics are available.Academic libraries can establish the value of works within IRs, encourage others to deposit works in IRs, locate what groups are using their content, and educate institutions on altmetrics (Bernal, 2013;Burns & Inefuku, 2015;Konkiel & Scherer, 2013;NISO, 2016;Roemer & Borchardt, 2015b).Authors (faculty, students) of the IRs' content can showcase the impact of their works in times of evaluation (e.g.grants, promotion and tenure) as well as identify potential collaborators and topics with high public interest (Burns andInefuku, 2015, Konkiel &Scherer, 2013;NISO, 2016).The IR platform Digital Commons has the ability to host journals edited and conferences organized by institutions' faculty, students, and/or administrators.These stakeholders can demonstrate the influence of their journals or conferences, focus on popular themes or topics, and encourage authors to contribute their research (NISO, 2016).Lastly, administrators can exhibit the impact of their institutions' scholarly outputs, whether to request budget increases from university trustees or recruit potential faculty and students (Burns & Inefuku, 2015;Konkiel & Scherer, 2013;NISO, 2016).They can also utilize altmetrics to evaluate which projects to fund, determine return on investments on that funding, and create benchmarks for groups on campus (Konkiel & Scherer, 2013).
These potential benefits represent the possibilities of altmetrics rather than the reality.The current study showcases a real world example of Twitter metrics that IR stakeholders would receive in order to better understand the practical benefits and uses of altmetrics.
Out of all the stakeholders, academic libraries are becoming the gatekeepers of metrics measuring institutions' scholarly output.Within the scholarly communications field, altmetrics have become a core competency for librarians and a popular topic in professional development (Cross, Oleen, & Perry, 2017;North American Serials Interest Group, 2017).Libraries have published on their experiences of integrating altmetrics with their IRs without delving into the systematic evaluation of altmetrics (Collister & Taylor, 2017;Wong & Vital, 2017).Even researchers selected IRs and libraries as the most trusted source of altmetrics (Zhang & Jackson, 2016).Given the interest among academic libraries and support among researchers, there is a need for more scholarship that explores the relationship between IRs and altmetrics.

Methods
The author conducted a content analysis of tweets extracted from Georgia Southern University's PlumX Metrics (https://plu.mx/georgiasouthern/g)mentioning works from four collections in their institutional repository (IR), Digital Commons@Georgia Southern (https://digitalcommons.georgiasouthern.edu/):journals edited by university faculty, conferences organized by university faculty, student research (i.e.undergraduate honors theses, graduate theses and dissertations), and faculty publications.The journals, conferences, and student research collections consisted of works originally published in the IR (or previous IR platform), while the faculty publications collection consisted of works originally published by outside publishers.The author coded information on the tweets' attitudes, authors, audience, hashtags, and URLs.Content analysis was chosen due to its flexibility in coding, specifically the inclusion of tweets with opinions unrelated to the works and expansion tweets as mentioned by Liu & Fang (2017).
The data in this study includes the tweets in Georgia Southern's PlumX Metrics and the works in Digital Commons@Georgia Southern (within the four collections) that were available on October 26, 2016.This includes any tweets and works published on or prior to this date.The The author collected the tweets' content and publication dates, the tweeted works' titles, associated collections, disciplines, and original publication dates, and the URLs to the PlumX Metrics and Digital Commons records.Three percent (290 of 9671) of the works were mentioned in tweets and eligible for analysis (see Table 1).The works mentioned in tweets were primarily published from 2013 to 2016 within the fields of education, medicine and health sciences, and social and behavioral sciences (see Table 2).Due to individual works having multiple tweets, 605 tweets were available for the analysis.The majority of tweets were posted from 2013 to 2016 and mentioned works within the faculty publications collection followed by journals, conferences, and student research collections (see Table 3).

Results
In terms of attitude for all tweets, 64.8 percent (n=392) were neutral, 33.4 percent (n=202) were positive, and 1.8 percent (n=11) were negative.The most common attitude subcategories were neutral tweets mentioning the works' title (48.4%), positive tweets expressing opinions about the works (15.5%), and neutral tweets paraphrasing the works (13.9%).The other subcategories represented 10.0 percent or less of the tweets.Unaffiliated individuals (51.7%) or organizations (22.8%) authored most of the tweets.Affiliated authors (13.7%), organizations (11.2%), and colleagues (0.5%) wrote approximately a quarter of the tweets.80.3 percent of tweets were meant for a general audience, with 12.5 percent mentioning specific user(s), and 7.8 percent mentioning user(s) affiliated with the work.
The majority of the tweets (64.1%) did not have hashtags.Title (31.8%), subject (25.8%), and outside topics (12.0%) were the most common hashtags with the remainder categories falling below 10 percent.Review Table 4 and Table 5 for comprehensive numbers and percentages of attitude, author, audience, and hashtag categories and subcategories based on all, neutral, positive, and negative tweets.

Neutral Tweets
The neutral tweets represented the majority with 392 tweets out of 605 tweets or 64.8 percent of the tweets.The neutral tweets overwhelmingly included the titles of the works (74.7%).
Approximately one-fifth (21.4%) paraphrased either the content or title of the works.Lastly, 12.5 percent included the author's name and 10.2 percent provided the conference or publication information.Other neutral attitude subcategories had percentages lower than 10.0 percent each.
Individuals (52.8%) and organizations (29.8%) unaffiliated with the works wrote the most tweets.The next highest were affiliated organizations (9.2%) followed by authors (8.2%).Most tweets (92.3%) were written for a general audience with no mention of other Twitter accounts.The majority of tweets (69.4%) did not have hashtags associated with them.Hashtags included keywords from the titles (37.3%) or subjects associated with the works (30.0%).Other categories had 10 percent or less each.
As for the audiences, 56.9 percent of the positive tweets were intended for a general audience, 31.2 percent were for a specific audience, and 13.9 percent were affiliated communication.Roughly half of the tweets (52.5%) did not have hashtags.Hashtags subcategories included outside topics (26.4%), titles (24.3%), and subjects (20.1%).The rest of the hashtags subcategories were less than 10 percent each.

Negative Tweets
Only 11 negative tweets existed out of 605 tweets, or 1.8 percent of tweets.The negative tweets primarily expressed concerns about the content of the work (81.8%) with a small percentage commenting on the price of the work (9.1%) and the act of reading the work (9.1%).The majority of the tweets were for a general audience (81.8%) by individuals with no affiliation to the works (90.9%) and had no hashtags (90.9%).

URLs
The tweets from the journals, conferences, student research collections included links to the institutional repository (IR).The tweets from the faculty publications collection included links to other websites where the works were published such as publisher websites, preprint databases, etc.The exception was one URL to a faculty member's profile within the IR.

Discussion
Despite the limited sample size, the results parallel other studies' findings including low percentage of tweeted works, high percentage of neutral attitudes in tweets, and evidence of selfmentions.The discussion expands on the meaning of these trends and other results specific to this study including the high percentage of tweets for a general audience, evidence of networking hashtags, and URLs leading to the institutional repository (IR) and original publishers' websites.The could be attributed to content analyses having broader coding categories for positive tweets.
Approximately half of the positive tweets from this study did not express opinions about the works and were comparable to neutral tweets' implicit messages of agreement or relevancy to their followers (Thelwall et al., 2013).
Since neutral tweets have demonstrated to be the most common attitude or sentiment, Twitter metrics would benefit from borrowing the trends of citation metrics that ignore sentiment or attitude (Thelwall et al., 2013).Liu and Fang (2017)  Unfortunately, discovering a tweet that expresses that information may be scarce due to the low numbers of tweets and high chances of neutral tweets.

Self-Mentions
In this study, roughly a quarter of the tweets were authored by individuals or organizations affiliated with the cited works.Self-mentions were more common in positive tweets than neutral and negative tweets.Other studies have noted their observation of self-mentions (Liu & Fang, 2017;Maleki, 2014;Ortega, 2016;Thelwall et al., 2013).Based on the commonality of self-mentions, separating tweets by affiliated individuals and organizations would be beneficial.For authors (faculty and students) and administrators, self-mentions could change the meaning of their metrics by indicating promotional effort rather than attention or visibility (Ortega, 2016).Authors and administrators may choose to extract self-mentions from their reports because of this representation of promotion.In contrast, academic libraries may use self-mentions as evidence that internal stakeholders are utilizing and interacting with their IRs.Given the different meanings for stakeholders, separating the self-conference.These hashtags represented discussions on a specific subject (chat), between a higher education course (class title), and during events unaffiliated with the work (outside conference).
Faculty and student authors who use Twitter for its networking capabilities would benefit the most from these hashtags (Veletsianos, 2011;Veletsianos & Kimmons, 2016).Unfortunately, this potential benefit would be unlikely as these categories represented only five percent of all hashtags.
Administrators and IR managers do not obtain as much value from hashtags, unless they have Twitter accounts to promote the institutions' or IRs' scholarly works.In these cases, institutional Twitter accounts could use the hashtags to communicate and promote the scholarship of the institution and IR to focused groups of Twitter users.

Repository vs. Publisher URLs
The tweets mentioning works originally published in the IR (journals, conferences, student research collections) linked to the IR.The tweets mentioning works originally published outside the IR (faculty publications collection) primarily linked to outside websites.It should be noted that metrics aggregators must have an identifier to track scholarly works.For example, presentations at conferences will not be tracked unless the tweets include certain identifier(s) (URLs, DOIs, etc.).If an attendee tweets about a presentation with the hashtag for the conference, but without the correct identifier, then it will not be captured.IR stakeholders need to consider the need for an identifier in order to accurately capture tweets about works in the IR.
The URLs' destinations may affect the stakeholders' perceived value of the tweets.Authors (faculty and students) and administrators would benefit from collecting tweets from every possible source, because they are all relevant in measuring attention on their affiliated works.In contrast, academic libraries may want to separate the tweets that link to IRs from those linking to other sources.
The former demonstrates the attention IRs receive as a publishing platform in contrast to demonstrating the impact of the institutions' scholarly works.Since altmetrics accumulate immediately after publication, IR managers will want to continue to request that authors include their works (preprints, postprints, presentations) in their IRs before the works are published on another website to increase the chances that the tweets will refer to the IRs' URLs (Holmberg et al., 2015).

Limitations
The results represent a single institution's data and cannot be applied broadly.Participation within the institutional is voluntary with the exception of the student research collection.The subject areas of the tweeted works were concentrated in education, medicine and health sciences, and social and behavioral sciences, which could have affected the types of tweets the study collected (Friedrich, Bowman, & Haustein, 2015;Vainio & Holmberg, 2017).

Conclusion
Institutional repositories (IRs) are the central hub for the scholarly output of higher education institutions, which means the entire campus has stakes in the success of the IR.IR stakeholders (faculty, students, administrators, and academic libraries) have varying motivations and purposes for using IRs.The integration of altmetrics with IRs can enhance the IRs' value to stakeholders by offering the ability to demonstrate impact.As managers of IRs, academic libraries can become responsible for the accuracy, promotion, and reporting of altmetrics across campus.This may entail changing metadata standards to ensure optimal integration with the altmetrics product, promoting the value of altmetrics on campus, instructing administrators and faculty how to report altmetrics, and creating reports for them.Therefore, altmetrics research with a focus on scholarly outputs in IRs are important to explore.
This study was successful in providing a snapshot of the tweets tracked by a metrics aggregator in connection to an IR.It helps inform how beneficial tweets may be for different stakeholders and ways to improve Twitter metrics.However, this study represents a miniscule portion of the major potential for altmetrics studies revolving around academic libraries and IRs.Further research needs to investigate the broad range of altmetrics (Mendeley, Facebook, Reddit, etc.) as well as produce a larger sample size by examining multiple IRs.Qualitative studies need to focus on the motivations of IR stakeholders.For example, interviewing or surveying stakeholders on their perception of altmetrics in IRs can further determine if the potential use cases have real world value.
The more altmetrics research focus on IRs, the more informed academic libraries, and consequently others on campus, will become on interpreting and utilizing altmetrics.
following within the tweet: excitement for attending a conference, congratulations for publishing, announcement of a new publication, positive opinion about the work, statement of reading/checking out a work, or recommendation to read the work.Negative tweets either expressed a negative opinion about the work's content, price, or the act of reading it.The author categories defined who wrote the tweets based on their relation to the work, including unaffiliated individuals and organizations, affiliated authors and organizations, and colleagues of the works' authors.The audience categories indicated if the tweets were intended for a general audience, specific Twitter user(s), or Twitter user(s) affiliated with the work.Lastly, it was noted whether or not hashtags (keywords following a pound symbol) were included in the tweets.Then, subcategories were created based on the content of the hashtags such as information about the work (author, author's affiliation, conference/publication, date, subject, and title) and other information (365 challenge, Twitter chat, class title, unaffiliated organization and topics, research, Twitter handle, and Twitter handle's affiliation).Consult Appendix A for descriptions and examples of the coding categories for attitude, authors, audience, and hashtags.

Table 1
Total Works Versus Tweeted Works by CollectionThe tweets were coded into three primary attitude categories (neutral, positive, negative) with more detailed subcategories.Neutral subcategories reflected information about the work mentioned in the tweets such as title, author, author's affiliation, title of conference or publication, date, reference to Digital Commons, paraphrase of a main idea, and/or quote.Positive subcategories expressed the

Table 2 1
Disciplines and Dates of Tweeted Works 1 N = 290.Dates do not correspond with the posting date in Digital Commons.For example, the student research collection includes digitized theses and dissertations whose original publication date represented in the table is earlier than the posted date in Digital Commons.

Table 3 2
Tweets by Collection and Date

Table 5 4
Hashtag Subcategories suggested excluding negative tweets and weighting certain positive tweets.However, these actions would not drastically change the metrics due to the low numbers of non-neutral tweets found in previous studies.Although this study had more positive tweets, distinguishing between different types of positive attitudes or sentiments would be difficult, especially with those that are similar to the implicit meaning of neutral tweets.Based on the high probability of neutral tweets, Twitter metrics indicate attention or visibility rather than high-levels of impact.Twitter metrics may be used to determine what subjects are receiving the most buzz in order to inform decisions on what to pursue for research projects (faculty and student authors), funding (administrators), or IR collections (academic libraries).In terms of qualitative data, positive and negative tweets have potential benefits for faculty and student authors, because they can pinpoint which part of the works garnered the most interest or criticism.