AI Meets AV in Higher Ed

Penn State University’s Bluebox, an experimental learning space driven by students. A touchscreen display allows multiple students to push content to the screen with ease. Every wall is a writeable surface. Floor-based power can appear when its needed and disappear when it’s not. “We don’t have one size fits all students, why should we build one size fits all classrooms?” (Image credit: Penn State University)

Will artificial intelligence (AI) trigger a dystopic future (re: Skynet)? Will machine learning help cure cancer? Theories abound in the AI conversation. What’s not debatable, however, is AI’s popularity. Consumers are adopting it at a rapid clip and manufacturers are investing heavily in AI research and development. More than 80 percent of executives surveyed by the MIT Sloan Management Review view AI as a strategic opportunity. In one form or another, it is advancing solutions for facial and speech recognition, language processing, and analytics. There’s also AI-enabled search, voice assistance, and digital services. Education technology stakeholders, in particular, are watching AI developments closely to learn where an infusion of machine intelligence can improve classroom UX. How can AI add value to “traditional” campus AV?

AERIAL VIEW

For a university CIO, it might not be obvious yet where AI, AV, and academia will intersect. Nonetheless, Rob Lipps, Sonic Foundry’s EVP, says now is the time to explore AI and how it can elevate the classroom experience. “More and more people are wondering how we can use AI on a campus,” Lipps said.

Video intelligence, analytics, and the recording, management, and delivery of video are natural fits for AI. Lipps sees the power of smart indexing and analytics every day with his company’s flagship video capture and management solution, Mediasite Video Platform. Mediasite has been adopted by schools and organizations across the globe.

PERSONALIZED LEARNING

AI has the potential to comb through tens of thousands of hours of academic video and create personalized results for learners. It can also gather more identifiable data about the people watching videos.

There’s a wrinkle in the AI scenario, however. “AI works best when you have a huge pool of data,” Lipps added. For AI in education to provide the most measurable benefits, the technology needs access to as many videos as possible. But AI at scale runs into compliance issues.

ACCESSIBILITY AND COMPLIANCE

Architects of university policy must consider what videos (lecture capture, for example) will be made public, and how or if students can be identified in the videos. Facial recognition and object recognition—areas where AI provides advantages—are complex. Some educational leaders worry that student faces will appear on videos without consent.

What’s more, if videos are not captioned, they often cannot be made public. If a school can’t afford to caption all the videos it produces, the financial model may not make sense.

While it is still a mostly human endeavor, AI is assisting in accessibility efforts with video captioning. IBM Watson Captioning utilizes AI to automate the process for live broadcasts and on-demand video.

As AI evolves, technologists, academic leaders, and administrators must work together to find the balance between accessibility and compliance. University policies need to ensure student privacy while considering what new technologies can best support diverse learning styles. EDUCAUSE is a good resource for tech managers interested in learning more about AI and accessibility policies.

AI-ENHANCED AUDIO AND SPEECH-TO-TEXT

While video tends to steal the headlines, audio is an ideal category for AI enhancement. Arguably, the most important component of any recording—in any conference room, event, or lecture hall—is the audio. Nine times out of 10, students will watch a video with lower resolution versus unintelligible audio.

“While we call it ‘video in education, the focus of academic video or lecture capture tends to be on the moving object, not the audio,” Lipps said. “But video always has an audio file with it.”

In a typical lecture hall AV integration, audio solutions will be placed at the lectern or podium. There may be lapel mics for instructors, amplified mics, beamtracking mics, or hanging choir mics to capture student questions in the back row. Installed audio endpoints may also record the ruffling of papers, casual chatter, coughs and sneezes, or HVAC rumblings. That noise introduces artifacts into an audio file; self-learning or machine-based AI captioning might be a solution for cleaning up compromised audio.

Cognitive AI-based tools, like IBM Watson, can also help tech managers learn what AV audio room configuration might best enhance the accuracy of AI-based speech-to text for indexing.

AI SUPPORTS TEACHING, LEARNING, AND OER

Penn State’s approach to AI is rooted in its dedication to supporting teaching and learning, said Kyle Bowen, director of education technology services, Penn State University. This passion for innovation inspired Penn State to create its experimental learning space, the Bluebox Studio, and BBookX, a way to develop textbooks using AI.

An obstacle to wider adoption of open educational resources—or open textbooks—is the availability of supplemental material such as tests and quizzes. Established textbooks often come packed with question banks and teacher guides. With an open textbook, or an independently created volume, additional pieces need to be built. “That’s where we recognized an opportunity to use AI as an assistant to help construct assessments,” Bowen said. “What it requires us to do is think about how we capture the totality of the conversation in the room—and not just the instructor.” He cited the placement of audio and video in the classroom to help record dynamic learning experiences with fidelity.

“From there, we take that recording and use a series of AI tools that can identify, or try to identify within reason, what kinds of activities are happening,” Bowen said. The team can then work to create valuable supplemental materials. The result: a living textbook.

AI AND PERSONALIZATION STUDENT RETENTION

As the Penn State example illustrates, AI can deepen engagement, and not just for students. It can encourage dynamism among students, the professor, and course materials. Students can quite literally contribute to the creation of a new textbook. The trick is to keep all interfaces streamlined.

While many variables determine whether a student will matriculate or not, some AI assistants, AI apps, and personalized learning plans are proving to be useful tools for student retention. Lipps cited an example: “Some students who watch the most hours of video might be struggling more.” The key is to “fit data points into a bigger picture,” he added, with context that fits the multiple modes of learning.

ACADEMIA SUPPORTS THE AI ECOSYSTEM

Academic institutions are playing integral roles in the emerging AI ecosystem—from being AI beta testers to nurturing the next talent pool of inventors. IBM, makers of IBM Watson’s cognitive AI suite and long-time tenant at the SUNY Polytechnic Institute campus in Albany, NY, plans to invest more than $2 billion to grow its footprint on the campus and throughout New York state. This includes building the IBM AI Hardware Center at SUNY Poly for AI-focused research, computer chip research, development, prototyping, testing, and simulation.

AI DECISION MAKERS ON CAMPUS

Who decides on emerging technology procurement in 2019? The answers will vary, but frequently there is a split in operations and budgets between the academic side of a school and the administrative side, perhaps working under the CIO. AV technology is often part of learning technologies division, under the governance of the provost. Compliance and accessibility teams are also in the academic division. In next-level lecture capture deployments, for example, will the compliance teams, IT, and AV technologists need to collaborate more frequently or in different ways? This is just one of the questions in the era of AI.

FUTURE FORECAST

As video adoption continues and more classes blend, flip, or go online, AI will undoubtedly influence the consumption of AV.

Lipps added: “Our goal is to make content more discoverable and organized. Imagine a student creating a playlist of videos using a specific search term. Now imagine delivering that functionality to a broader community.”

Margot Douaihy is the editor-at-large of AV Technology magazine. She teaches at Franklin Pierce University.