Memory Network Approach

Jeff 06/27/2018

A discrete input set $\bar{x}_1,\cdots,\bar{x}_n$, are paragraph.

The paragraph will be transformed to memory $\bar{c}$ and $\bar{m}$.

A query $q$ is the input to the model.

$q=[\bar{q}_1, \bar{q}_2,\cdots, \bar{q}_{J}]$, where $\bar{q}_1$ is the first word in query, $\bar{q}_1,\bar{q}_2,\cdots,\bar{q}_J$ are 1-N encoding.

The answer $a$ is the output of the model.

Each of the $x_i$, $q$ and $a$ contains symbols coming from a dictionary with $V$ words.

The memory $x$ has fixed buffer size.

The continuous representation for the $x$ and $q$ are processed via multiple hops to output $a$.

This allows error signal back-propagation through multiple memory accesses back to the input during training.