Import modules in utility.py file
import numpy as np
from numpy import sum
from random import shuffle
import operator
import json
import csv
from pprint import pprint
numpy, sum, suffle, operator, json, csv and pprint modules are imported.
batchPadding function in utility.py file
def batchPadding(batchSentence,max_len):
""" Padding batch of question or choices
Parameters
----------
batchSentence : 3d numpy array, 3rd level list
each dimension represents (batch, sentence, word_vec) respectively
max_len : int
length of the longest sentence (default is 50)
Returns
-------
3d numpy array
batch of question or choices (padded)
"""
newBatchSentence = []
filler = [0]*300
for sen in batchSentence:
newBatchSentence.append(sen + [filler]*(max_len-len(sen)))
return np.asarray(newBatchSentence)
Fig.1 Schematic of batchPadding function.
Fig.1 shows the schematic of batchPadding function with "batchSentence" (3d numpy array) and "max_len" (integer) as inputs.
The "newBatchSentence" is appended overall "batchSentence", and "newBatchSentence" is returned.