Ulatus provides Abstractive and Extractive AI + Human summarization solutions to Coursera.

Ulatus summarizes 200+ videos into high-level, conceptual, & concise summaries through its AI + Human Extractive and Abstractive Summarization services.

Thanks to you and your team for getting the job done so efficiently!

Project Summary

Coursera AI + Human summarization solutions
  • Service Name: Summarization
  • Language Pair: English to English
  • Subject Area: Machine Learning, Programming, and Business Management
  • Volume: 35 hrs. / 300 thousand words
  • Link: https://www.coursera.org/

About the company, what were their requirements and why?

Coursera is an online learning platform founded by two Stanford University computer science professors. It offers thousands of online courses in partnership with over 200 of the world's leading universities and companies, including Yale, Princeton, Penn, Stanford, Google, and more. Based on learner feedback, Coursera identified the need for video summarization for their courses especially when learners are familiar with the course and don’t want to listen to the entire video. Also, this could serve as a refresher for learners in the future who want to go back and do a quick review of the course.

Challenges we faced during the project

Aggressive Timelines and Lack of Guidelines

The client had given us only 3 weeks to complete the whole project. To add to the complexity, the client did not have any rule of thumb as far as the length of the summarization was concerned, as the summary length was largely dependent on the lecture in question. Coursera deferred to us to make the right judgments, as long as the summaries stuck to their original requirements of being concise and high-level.

Technical courses of varying lengths and informal language

Most of the courses were technical in nature and jargon-heavy. The summary had to be such that gives an overview of the high-level concepts, keywords, and definitions (eg. What is a decision boundary? How can we make a decision boundary more complex?) while avoiding contextualized examples. Some lectures also used informal language that had to be avoided in a concise bulleted summary.

Tracking so many summaries was extremely difficult

The client was going to review all the summaries with the help of 3-4 internal reviewers and it was imperative that a tracking mechanism (communal format) was put in place for both teams to be on top of all the deliveries - before and after review. All changes and suggestions then had to be incorporated again by our summarizers.

Solution provided by us to the client-

Proactively creating custom guidelines/style guides for both Abstractive and Extractive summaries

Based on the trial submission and output, our quality team and subject matter expert summarizers got on a call with the client and laid down the guidelines that needed to be followed. The guidelines were prepared to keep in mind what was expected from us and how the end audience would consume the content. This was the most important step from a market success perspective for the client.

Smart usage of technology to manage consistency and overall quality of the project

We trained our proprietary summarization tool to meet the client's guidelines for both abstractive and extractive summaries such that the human summarizers got a solid base to work on further. Over and above the summarization done by our AI tool, we added another round of proofreading for the abstractive summaries with the help of Trinka AI, our proprietary grammar checker tool. Trinka's AI grammar correction tool brought about the accuracy, style guide preferences, conciseness, and contextual spelling checks.

A comprehensive QA process devised to achieve consistency in the summaries

With around 5‒6 summarization experts working simultaneously across multiple courses, around 10 interim batches were delivered during the project lifecycle. Each batch went through various review stages including Trinka QA and in-house quality QA. All these efforts helped us achieve our goal of delivering high-quality and consistent summaries.

Planning and scheduling to perfection

A schedule was created to deliver the summaries in a staggered manner such that both teams get equal time to review and finalize each summary. A tracking mechanism on Google Sheets was created for both teams to easily check the status of each summary. The files were directly uploaded onto the clients' Google drive location and were renamed following their exact specifications.

Let's Work Together