While the previous chapter discussed the development of Building LLTDM, in this part we discuss the detailed day-to-day activities and delivery of the virtual Institute. We also explain our post-Institute reconvening, and describe how we turned the Institute’s literacies and pedagogy into an open educational resource for broad dissemination.

Day by Day Institute delivery

This chapter will explain how we delivered the four day online Institute. As we already mentioned, due to the Covid-19 pandemic, we had to move the Institute from a planned in-person event to a fully remote, online experience. We kept the overall length of the Institute the same as we had planned for the in-person event, although the days were somewhat shortened to take into account the fact that participants and faculty were joining from different time zones, thus we wanted to ensure that we completed by end of normal business hours in the latest time zone (Eastern time). We typically ran the online version of the Institute beginning at 8am Pacific time and ending by 2pm Pacific time. We delivered the Institute in a flipped format (with readings and short videos prepared beforehand), so by ending by 2pm Pacific each day, it would provide time for participants to read and view the following days’ content, either later the same day for Pacific or Mountain time zone participants, or the following morning before we began again for Central and Eastern time zone participants.

Day 1

Introductions and setting the stage for the week

The faculty instructors used a primary slide deck throughout the course of the week. Day 1 began with a welcome, logistical information, and framing for the week’s activities.

One of the faculty instructors served as a moderator for the Institute. The moderator’s role was to observe and synthesize emerging themes from each day. The moderator helped bolster learning outcomes for participants and assist with cross pollination of ideas and themes from across small breakout groups. The moderator observed different groups of discussion sessions and collected individual reflections for sharing at the end of each day.

Empathy building exercise

Participants engaged in a virtual white board exercise designed to help them reflect on their own experiences with text data mining, to build knowledge and understanding among participants, and to surface aspects of divergence and convergence across individual experiences. We used the online “sticky note” software tool called Mural for this journey mapping exercise.

Free write

Day 1 ended with a free write exercise (the first of three over the course of the week). Free write time wasn’t intended for recapturing any notes participants took over the course of the day, but to reflect on the day’s sessions and apply them to their personal circumstances: their research interests, institutional culture, team dynamics, etc.

Participants were asked to write for 15 minutes straight without pausing or proofreading. We offered a few prompts to get them writing:

• What did you learn from other participants today about variations in TDM processes and logistical complexities?
• Which pain points highlighted by other participants resonated with you?
• What new questions, concerns, or opportunities emerged during report outs that you didn’t capture on the mural board?

At the end of the free write time, participants were asked to email their free write text to our shared faculty email group. Then, a small group of faculty instructors and the moderator reviewed responses each evening and discussed the day’s events in preparation for an opening reflection to kick off the next day.

At the end of day 1, faculty and participants were invited to an informal (and optional) “Happy Half-Hour” on Zoom. This time was to socialize, decompress, and answer questions.

Day 2

Report back from moderator on free write themes

At the beginning of day 2, the moderator summarized the themes and learnings that were communicated in the previous day’s free writes. This practice reminded participants about the themes discussed in the day before, and tracked progress and accomplishments over the course of the week.

Substantive literacies: Copyright, international copyright, TPMs

On day 2, we began to explore the substantive law and policy literacies for text data mining in the digital humanities. We covered copyright (focusing heavily on U.S. law), copyright in the international/cross-border context, and technological protection measures. As mentioned above, participants were able to watch short pre-recorded videos made by the faculty, as well as view slides and video transcripts.

Participants were asked to read the following articles in advance of day 2:

• Authors Guild v. Google, Inc., 804 F. 3d 202 – Court of Appeals, 2nd Circuit 2015, https://law.justia.com/cases/federal/appellate-courts/ca2/13-4829/13-4829-2015-10-16.html
• Matthew Sag, “The New Legal Landscape for Text Mining and Machine Learning,” SSRN Electronic Journal, 2019, https://doi.org/10.2139/ssrn.3331606
• Flynn, Sean and Geiger, Christophe and Quintais, João and Margoni, Thomas and Sag, Matthew and Guibault, L. and Carroll, Michael W., Implementing User Rights for Research in the Field of Artificial Intelligence: A Call for International Action (April 20, 2020). European Intellectual Property Review 2020, Issue 7. Available at SSRN: https://ssrn.com/abstract=3578819

“Putting it together”

After the morning substantive sessions, faculty and participants engaged in a “putting it together” exercise. This activity required individual reading and reflection, as well as small- and medium-sized group discussions, on a pre-prepared TDM scenario.

Free write

Day 2 ended with another free write exercise.

Participants were asked to write for 15 minutes straight without pausing or proofreading. We offered a few prompts to get them writing:

• How do the projects you’ve worked on, supported, or encountered differ from the scenario you worked on during the Putting it Together session?
• What copyright concerns do you have about accessing data for your own projects?  What about publishing it?
• What was your biggest “Ah ha!” moment of the day?  What do you still find confusing?

At the end of the free write time, participants were asked to email their free write text to our shared faculty email group. Then, a small group of faculty instructors and the moderator reviewed responses each evening and discussed the day’s events in preparation for an opening reflection to kick off the next day.

At the end of day 2, faculty and participants were invited to an informal (and optional) “Happy Half-Hour” on Zoom. This time was to socialize, decompress, and answer questions.

Day 3

Report back from moderator on free-write themes

At the beginning of day 3, the moderator summarized the themes and learnings that were communicated in the previous day’s free writes. This practice reminded participants about the themes discussed in the day before, and tracked progress and accomplishments over the course of the week.

Substantive literacies: Licensing, privacy & ethics

On day 3, we explored the substantive law and policy literacies for text data mining having to do with licensing, privacy, and ethics. Participants were able to watch short pre-recorded videos made by the faculty, as well as view slides and video transcripts.

Participants were asked to read the following articles in advance of day 3:

• California Digital Library 2005 Agreement with Factiva: https://cdlib.org/services-groups/collections/licensed_resources/redacted_licenses/ST_Tier2_Factiva_UCLA_2005_Redacted.pdf
• California Digital Library New Model Agreement: http://ucblib.link/33L
• Nancy Herther, Daniel Dollar, Darby Orcutt, Alicia Wise, and Meg White, “Text and Data Mining Contracts: The Issues and Needs” (2015). Proceedings of the Charleston Library Conference. http://dx.doi.org/10.5703/1288284316233
• Butler, Brandon (2018), “For text- and data-mining, fair use is powerful, but possession is still 9/10 of the law” at http://thetaper.library.virginia.edu/2018/02/28/for-text-and-data-mining-fair-use-is-powerful-but-possession-is-still-9-10-of-the-law-sparc.html
• Suomela, Todd, et al. “Applying an Ethics of Care to Internet Research: Gamergate and Digital Humanities.” Digital Studies/Le Champ Numérique, vol. 9, no. 1, Open Library of Humanities, Feb. 2019, p. 4, https://www.digitalstudies.org/articles/10.16995/dscn.302/
• Jules Polonetsky et al., Benefit-Risk Analysis for Big Data Projects, Future of Privacy Forum, (Sep. 2014), https://fpf.org/wp-content/uploads/FPF_DataBenefitAnalysis_FINAL.pdf 

“Putting it together”

After the morning substantive sessions, faculty and participants engaged in a “putting it together” exercise. This activity required individual reading and reflection, as well as small- and medium- sized group discussions, on a pre-prepared TDM scenario.

Free write

Day 3 ended with the final free write exercise.

Participants were asked to write for 15 minutes straight without pausing or proofreading. We offered a few prompts to get them writing:

• What strategies will you use to evaluate the ethical implications of current and future TDM projects?
• What licensing issues surfaced for your own work? Where do you see a path forward and where do you feel stuck?
• What made you feel angry today?  What made you feel relieved?

At the end of the free write time, participants were asked to email their free write text to our shared faculty email group. Then, a small group of faculty instructors and the moderator reviewed responses each evening and discussed the day’s events in preparation for an opening reflection to kick off the next day.

At the end of day 3, faculty and participants were invited to an informal (and optional) “Happy Half-Hour” on Zoom. This time was to socialize, decompress, and answer questions.

Preparation for implementation mapping discussion

Before Day 4, we asked the participants to read and reflect on the following questions at three implementation levels: As to their own practice, within their institution, and within their community. “Community” may refer to other digital humanities professionals and researchers with whom you interact, or any relevant broader group of stakeholders.

1. How will you provide guidance to others or integrate the literacies in your own practice? What concrete steps or actions will you take? Are there things that you, your institution, or the broader community should stop doing?
   1. Yourself:
   2. Your institution:
   3. Your community:
2. What challenges might you face as you move forward with implementation of the literacies?
   1. Yourself:
   2. Your institution:
   3. Your community:
3. How would you like to collaborate with other Building LLTDM participants or other DH researchers / professionals to integrate the literacies into DH TDM practice? What would a high level roadmap look like to achieve this vision? What support or funding would you need to make this vision possible?
4. Are there aspects of the current legal landscape that would benefit from community cooperation and advocacy to better address and enable TDM research?

Day 4

Report back from moderator on free-write themes

At the beginning of day 4, the moderator summarized the themes and learnings that were communicated in the previous day’s free writes. This practice reminded participants about the themes discussed in the day before, and tracked progress and accomplishments over the course of the week.

Implementation mapping

Faculty and participants reconvened in small groups to discuss the answers to the implementation mapping questions that they’d thought about the night before. These groups worked to identify common themes, next steps, needs, and plans. Later on, we all came together in a final plenary session to share take-aways from the small group discussion. We again used Mural to share virtual “sticky notes” that were viewable by all participants. Finally, we shared stickies of “gratitude” to acknowledge or thank participants, faculty, or recognize a particularly useful or impactful aspect of the Institute.

Participant evaluation

During the next-to-last session, we had participants fill out an evaluation survey. We hoped to get feedback right away while the participants would still have the Institute fresh in their minds, and so they didn’t have to respond to an email days or weeks later. We reminded participants that we would be coming back together in eight months for a meeting to update each other on our progress.

We wrapped the Institute with a short social celebration and goodbye on Zoom.

Institute reconvening & updates

The project team scheduled a check-in meeting with the participants eight months after the completion of the online Institute. The goal of the meeting was to see how participants had been thinking about, performing, or supporting TDM in their home institutions and projects with the LLTDM literacies in mind. In order to get participants thinking about what they wanted to report back to the group based on their experiences in the interim, we asked that they record a 2-minute video and post it on our Slack. In the video, we urged participants (and also faculty, if they had updates) to share their thoughts on the following:

• What have you been thinking about or doing with respect to TDM?
• What’s one lasting LLTDM lesson you remember from the Institute?
• What takeaways from the Institute have you been able to implement or share with others?
• What are you still struggling with when it comes to LLTDM?
• What are you proud of with respect to your LLTDM skills?

We asked that each person view all the other posted short videos. This way, they could get up to speed in a relatively short amount of time, and we wouldn’t have to spend a lot of time during the meeting itself giving individual updates.

The virtual meeting consisted of a welcome and reflections from the moderator based on the participant and faculty 2-min videos, small group discussions, and a plenary group exercise to discuss themes that emerged from the smaller discussions.

The last ask of the participants entailed completing a survey that asked about their TDM literacies implementation plans, and also gathered information about the types of resources that would be most useful to include in the open educational resource that would be published.