pandasToBrat

Ali BELLAMINE - contact@alibellamine.me Last version : 1.0 - 28/10/2020

What is pandasToBrat ?

pandasToBrat is a library to manage brat configuration and brat data from a Python interface.

### What can it do ?

Reading brat annotations and relations configuration to python dictionnary
Writting brat annotations and relations configuration from python dictionnary
Reading brat text data to python pandas dataframe
Writting brat text file from python pandas Series
Reading brat annotations and relations
Writting brat annotations and relations from python pandas DataFrame

What it doesn't support ?

Keyboard shortcut configuration
Event, Attribution, Modification, Normalization and Notes annotations
Relation type in relations configuration

How to use it ?

Installation

Clone the current repository :

    git clone [LIBRARY_PATH]

Install dependencies with pip.

    pip -r requirements.txt

Then install the library :

    pip install -e .

Loading a brat folder

Instantiate the brat library with the folder path :

    from pandasToBrat import pandasToBrat

    brat_data = pandasToBrat(FOLDER_PATH)

Parameters

Parameters are stored in a dictionnary :

    {
        "entities":ENTITIES_CONFIGURATION_DATA,
        "relations":RELATIONS_CONFIGURATION_DATA
    }

Entities configuration data

Dictionnary formated as :

    {
        LABEL_NAME:{
            LABEL_NAME_CHILD1:True,
            LABEL_NAME_CHILD2:True,
            LABEL_NAME_CHILD3:{
                LABEL_NAME_CHILD3_CHILD1:True
            }
        }
    }

Each entry is an entitie. An entitie can either be setted as True, it have no child, or have on or many childrens in which case is contains a dictionnary.

Relations configuration data

Dictionnary formated as :

    {
        RELATION_NAME:{
            "args":[ENTITIES_NAME,...]
        }
    }

Each entrie of the dictionnary is a relation. Each relation have a relation name and defined with a sub-dictionnary containing an args entrie. The args entrie contains a list of entities that are concerned by the relation.

#### Read and write parameters

Getting parameters

You can read the current parameters using the dedicated method :

    bratData.read_conf()

Writtings parameters

You can write parameters using the dedicated method :

    bratData.write_conf(entities = ENTITIES_CONFIGURATION, relations = RELATIONS_CONFIGURATION)

The ENTITIES_CONFIGATION is a dictionnary formated as described in the "Entities configuration data" chapter.

The RELATIONS_CONFIGURATION is a dictionnary formated as described in the "Relations configuration data" chapter.

Text

Text is stored in a Pandas Dataframe with two columns :

id : document id, which is contained in the .txt filename
text_data : document data

#### Read and write text

Getting text data

    bratData.read_text()

Sending text data

    bratData.write_text(text_id=TEXT_ID_SERIES, text = TEXT_SERIES, empty = EMPTY_PARAMETER, overWriteAnnotations = OVERWRITE_ANNOTATIONS_PARAMETERS)

The required parameters are text_id and text which are Pandas Series, which should be of the same size containing for the first one the document unique id and the second one the document text data.

The empty parameters is used to empty the current folder. If set as True, the Brat folder is emptied of all text and annotations data. Configuration is not erased.

The overwrite annotations parameter is used to overwrite the current annotation (.ann) file with an empty one, it is useful if you want to remove the existing annotations when you are modifiying a text file.

This way, you can :

Overwrite all data with empty set as True
Only overwritting new data with empty set as False and overWriteAnnotations set as True : you write new file, if the id already exist it is overwritten, if not is it ignored.

Annotations

Parameters are stored in a dictionnary :

    {
        "annotations":ANNOTATIONS_ANNOTATIONS,
        "relations":RELATIONS_ANNOTATIONS
    }

Annotations format

Annotations are word labeled with entities.

It is formatted as a Pandas DataFrame, containing the following columns :

id : Document id, one document can have mutiples annotations
type_id : annotation number inside the same document, from T1 to Tn, with n the number of annotated string, it is used to match annotations with relations
word : the annotated string
label : the entitie related to the annotated string
start : the annotated string start offset
end : the annotated string end offset

Relations format

Annotations are relations between annotations.

It is formatted as a Pandas DataFrame, containing the following columns :

id : Document id, one document can have mutiples relations
type_id : relation number inside the same document, from R1 to Rn, with n the number of relations
relation : The relation Name
ArgX : The annotated entitie which a linked by the relation, each column refer to an entitie, the entitie id correspond to the annotations DataFrame "type_id" column

#### Read and write annotations

Getting annotations data

    bratData.read_annotation()

Sending annotations data

Write annotations subpart of annotations

    bratData.write_annotations(df, text_id, word, label, start, end, overwrite=OVERWRITE_OPTION)

The first parameter is the datafame containing the annotations. It should be formated as described in the "Annotations format" subpart.

The text_id, word, label, start and end are the name of the column inside the dataframe which contains the related data.

The overwrite option can be set as True to overwrite existing annotations, otherwise the dataframe's data are added to existing annotations data.

Write relations subpart of annotations

    bratData.write_relations(df, relation, overwrite=OVERWRITE_OPTION)

The first parameter is the datafame containing the relations. It should be formated as described in the "Relations format" subpart.

The text_id and relation are the name of the column inside the dataframe which contains the related data. The other columns should contains the type_id of related entities, as outputed by the read_annotation method.