NeuralSight Official Documentation

Add Medical files to the workspace for preprocessing:

python preprocess.py

1. - Dropping images with `No findings` label: The first step in preprocessing the data was to drop all images with their label information that had a label called "No findings" because the label had no any bounding box information which were needed for model training. This was done to ensure that the model only focuses on images that contain the other 13 Chest X-ray pathologies of interest.
2. - Removing overlapping bounding boxes which occurs in the same class using Non-maximum suppression methods: In order to improve the performance of the model, non-maximum suppression was used to remove overlapping bounding boxes which occurs in the same class label. This technique helps to reduce the number of false positives by removing bounding boxes that have a high overlap with other boxes of the same class. The method worked as follows.
3. - Non-maximum suppression(NMS) function is used for removing overlapping bounding boxes. The function takes in two inputs: "boxes" and "overlapThresh". The "boxes" is a list of bounding boxes in the format of (x1, y1, x2, y2) where (x1, y1) is the top-left corner and (x2, y2) is the bottom-right corner. "overlapThresh" is a threshold value for the overlap ratio of bounding boxes.
4. - The function first checks if the input "boxes" is an empty list. If it is, it returns an empty list. If there are boxes, the function extracts the x and y coordinates of the top-left corner and bottom-right corner of each box. It then computes the area of each bounding box and sorts the boxes based on the bottom-right y-coordinate. Next, the function iterates through each box and compares it with the other boxes to calculate the ratio of overlap. If the overlap ratio is greater than the threshold, the function removes that box from the list of boxes to be returned. Finally, the function returns the remaining boxes as integers.
5. - Bounding Box information Normalization: The bounding box information were normalized to standard values between 0 and 1 against the width and height of the image. This ensures that the bounding box coordinates are consistent regardless of the size of the input image that a user will feed to the model. It allows us to work on any image size available.
6. - Calculating the height and width of the bounding box since they are one of the crucial features that must be fed to the model during training, they are used to identify where what is the size of the box.
7. - Training and Validation. Before Training, We split the data input two in order to validate the model after training. We used `GroupKFold method to split`. This method is a variation of the KFold method for cross-validation. It is used to split the data into `k` number of folds, where `k` in this case, we used k= 5 which means the data will be split into 5 folds.

                                from tqdm.notebook import tqdm
                                from glob import glob
                                import shutil, os
                                import cv2
                                import pandas as pd
                                import numpy as np
                                from cleaning import create_data

                                import warnings
                                warnings.filterwarnings("ignore");

                                import dotenv
                                dotenv.load_dotenv()

                                DATA_DIR = os.getenv("DATA_DIRECTORY")
                                TRAIN_DIR = os.getenv("TRAIN_DIR")
                                WANDB_API_KEY = os.getenv("WANDB_API_KEY")


                                #check if the file exists
                                if not os.path.isfile(f'{DATA_DIR}/train.csv'):
                                    print(f"No SUch Directory in the path Passed  : PATH SEND :: {DATA_DIR}/train.csv")
                                    print("Exiting....")
                                    exit(0)
                                # read the data
                                df = pd.read_csv(f'{DATA_DIR}/train.csv')

                                df1 = df[["image_id","x_min","y_min","x_max","y_max","class_id", "height", "width", "class_name"]].dropna()
                                df1['path'] = df1['image_id'].apply(lambda x: f"{DATA_DIR}/train/{x}.png")

                                # get class id
                                class_dict = {v:k for k,v in dict(df1[['class_name', "class_id"]].values).items()}


                                # save  df1 in order to use it for plotting
                                df1.to_csv("train_clean.csv", index=False)


                                check_df = create_data(df1)
                                # class names available
                                classes = list(check_df['class_name'].unique())




                                # Yolo normaly requires bbox to be normalized between 0 and 1.
                                # Since we have have height and width for a particular image, we gonna just divide bbox against either widht or height.
                                # We then extract the widht and height of the bbox from the above calibrations


                                df = check_df.copy()

                                df['x_min'] = df.apply(lambda x: (x.x_min)/x.width, axis =1)
                                df['y_min'] = df.apply(lambda x: (x.y_min)/x.height, axis =1)

                                df['x_max'] = df.apply(lambda x: (x.x_max)/x.width, axis =1)
                                df['y_max'] = df.apply(lambda x: (x.y_max)/x.height, axis =1)
                                df['x_mid'] = df.apply(lambda x: (x.x_max+x.x_min)/2, axis =1)
                                df['y_mid'] = df.apply(lambda x: (x.y_max+x.y_min)/2, axis =1)
                                df['w'] = df.apply(lambda x: (x.x_max-x.x_min), axis =1)
                                df['h'] = df.apply(lambda x: (x.y_max-x.y_min), axis =1)

                                df['area'] = df['w']*df['h']


                                selected_classes = (df['class_name'].value_counts(normalize=True).T[df['class_name'].value_counts(normalize=True).T >0.0001])
                                SELECTED_CLASS_NAMES  = list(selected_classes.index)

                                # Get a dataframe with selected classes
                                selected_df = df[df['class_name'].isin(SELECTED_CLASS_NAMES)]


                                # only which have below 40 bbox instances
                                img_ids = list((df['image_id'].value_counts(normalize=False).T[df['image_id'].value_counts(normalize=False).T <=40]).index)


                                # pd.DataFrame(base_details)

                                TRAIN =[]
                                # for img_id in selected_df['image_id'].unique():
                                for img_id in selected_df[selected_df['image_id'].isin(img_ids)]['image_id'].unique():
                                    curr_df = selected_df[selected_df['image_id'] ==img_id].reset_index(drop=True)
                                    base_details = dict(curr_df.loc[0][['image_id',"path",'width', 'height']])
                                    information =[]
                                    for indx in range(curr_df.shape[0]):
                                        other_details = dict(curr_df.loc[indx][['class_name', "x_min", "y_min","x_max","y_max", "x_mid", "y_mid", "w", "h", "area" ]])
                                        information.append(other_details)

                                    TRAIN.append([base_details['image_id'],base_details['path'] ,base_details['width'],base_details['height'],information])


                                final_data = pd.DataFrame(TRAIN, columns =['image_id',"path", "width", "height", "information"])
                                from pprint import pprint
                                print(final_data.head())
                                # save this data for feature usage.
                                final_data.to_csv("processed.csv", index=False)
                            

                                1. `/WORKING_DIR/PROJECT_DIR/labels/train` - This directory will store the labels (annotations) for the training images.
                                2. `/WORKING_DIR/PROJECT_DIR/labels/val` - This directory will store the labels (annotations) for the validation images.
                                3. `/WORKING_DIR/PROJECT_DIR/images/train` - This directory will store the training images.
                                4. `/WORKING_DIR/PROJECT_DIR/images/val` - This directory will store the validation images.
                            

Once the directories were created, We then moved or copy the images and their corresponding labels to the appropriate directories. The labels for each image should have the same file name as the image, and should be placed in the appropriate labels directory (either `/WORKING_DIR/PROJECT_DIR/labels/train` or `/WORKING_DIR/PROJECT_DIR/labels/val`). The labels were also in their own format. Each image had a single text file as labels. The format of labels for YOLOv5 model used was a plain text file with the same name as the corresponding image file, and with a `.txt` extension. The file should contain one line for each object in the image, with the following format:

                                  ````
                                    class_id x_center y_center width height
                                  ````
                            

where:

• class_id: An integer representing the class of the object. This should correspond to the class labels in your dataset.
• x_center: The x-coordinate of the center of the bounding box, as a fraction of the image width.
• y_center: The y-coordinate of the center of the bounding box, as a fraction of the image height.
• width: The width of the bounding box, as a fraction of the image width.
• height: The height of the bounding box, as a fraction of the image height.

Each object should be split in a new line. Here's an example of a label file for an image with two objects, a Pneumothorax and a Cardiomegaly:

                                ````
                                  12 0.5 0.5 0.2 0.2
                                  3 0.7 0.7 0.1 0.1
                                ````
                            

This means that there is one Pneumothorax object with class label 12, center coordinates (0.5,0.5), width and height as 0.2 and one Cardiomegaly object with class label 3, center coordinates (0.7,0.7), width and height as 0.1

                               ````yaml
                                  names:
                                  - Cardiomegaly
                                  - Pleural effusion
                                  - Pleural thickening
                                  - Aortic enlargement
                                  - Pulmonary fibrosis
                                  - ILD
                                  - Nodule/Mass
                                  - Other lesion
                                  - Lung Opacity
                                  - Infiltration
                                  - Consolidation
                                  - Calcification
                                  - Atelectasis
                                  - Pneumothorax
                                  nc: 14
                                  train: /kaggle/working/train.txt
                                  val: /kaggle/working/val.txt
                                ````
                            

where:

• - train and val is a text file with all directories for images used for training and validation respectively
• - nc is the number of classes
• - names contains all class names