Chef Recipe | Extractive summarization with Azure Text Analytics

Call graph

Call graph

---

Configuration

Prerequisites: - Add the .env file in the same folder of the notebook

Parameters

gist_user = 'davidefornelli'
gist_chef_id = '1bc116f05d09e598a1a2dcfbb0e2fc22'
gist_ingredients_id = '5c75b7cdea330d15dcd93adbb08648c3'
ingredients_to_import = [
    (gist_ingredients_id, 'az_cs_summarization.py')
]

texts = [
    '''
        A computer is a machine that can be programmed to carry out sequences of arithmetic or logical operations automatically. Modern computers can perform generic sets of operations known as programs. These programs enable computers to perform a wide range of tasks. A computer system is a "complete" computer that includes the hardware, operating system (main software), and peripheral equipment needed and used for "full" operation. This term may also refer to a group of computers that are linked and function together, such as a computer network or computer cluster.
        A broad range of industrial and consumer products use computers as control systems. Simple special-purpose devices like microwave ovens and remote controls are included, as are factory devices like industrial robots and computer-aided design, as well as general-purpose devices like personal computers and mobile devices like smartphones. Computers power the Internet, which links hundreds of millions of other computers and users.
        Early computers were meant to be used only for calculations. Simple manual instruments like the abacus have aided people in doing calculations since ancient times. Early in the Industrial Revolution, some mechanical devices were built to automate long tedious tasks, such as guiding patterns for looms. More sophisticated electrical machines did specialized analog calculations in the early 20th century. The first digital electronic calculating machines were developed during World War II. The first semiconductor transistors in the late 1940s were followed by the silicon-based MOSFET (MOS transistor) and monolithic integrated circuit (IC) chip technologies in the late 1950s, leading to the microprocessor and the microcomputer revolution in the 1970s. The speed, power and versatility of computers have been increasing dramatically ever since then, with transistor counts increasing at a rapid pace (as predicted by Moore's law), leading to the Digital Revolution during the late 20th to early 21st centuries.
        Conventionally, a modern computer consists of at least one processing element, typically a central processing unit (CPU) in the form of a microprocessor, along with some type of computer memory, typically semiconductor memory chips. The processing element carries out arithmetic and logical operations, and a sequencing and control unit can change the order of operations in response to stored information. Peripheral devices include input devices (keyboards, mice, joystick, etc.), output devices (monitor screens, printers, etc.), and input/output devices that perform both functions (e.g., the 2000s-era touchscreen). Peripheral devices allow information to be retrieved from an external source and they enable the result of operations to be saved and retrieved.
    '''
]

Configure environment

%pip install httpimport python-dotenv

Requirement already satisfied: httpimport in /home/daforne/repos/github/davidefornelli/cookbook/.venv/lib/python3.7/site-packages (0.7.2)
Requirement already satisfied: python-dotenv in /home/daforne/repos/github/davidefornelli/cookbook/.venv/lib/python3.7/site-packages (0.19.2)
Note: you may need to restart the kernel to use updated packages.

Import chef

import httpimport

with httpimport.remote_repo(
    ['chef'],
    f"https://gist.githubusercontent.com/{gist_user}/{gist_chef_id}/raw"
):
    import chef

Import ingredients

def ingredients_import(ingredients):
    for ingredient in ingredients:
        mod, package = chef.process_gist_ingredient(
            gist_id=ingredient[0],
            gist_file=ingredient[1],
            gist_user=gist_user
        )
        globals()[package] = mod


ingredients_import(ingredients=ingredients_to_import)

Extract summaries

import os
from dotenv import load_dotenv
load_dotenv()


# Apply summarization
summary_text = az_cs_summarization.summarize(
    texts=texts,
    cs_endpoint=os.environ['CS_TEXTANALYTICS_ENDPOINT'],
    cs_key=os.environ['CS_TEXTANALYTICS_KEY'],
    language='en'
)

Results

for sx in summary_text:
    for s in sx.sentences:
        print(s.text)

A computer is a machine that can be programmed to carry out sequences of arithmetic or logical operations automatically.
These programs enable computers to perform a wide range of tasks.
A broad range of industrial and consumer products use computers as control systems.