An Introduction to Deep Learning for the Physical Layer

Abstract

Several novel applications of deep learning for the physical layer were present. By interpreting a communications system as an autoencoder, a fundamental new way was developed to think about communications system design as an end-to-end reconstruction task that seeks to jointly optimize transmitter and receiver components in a single process.

The idea can be extended to networks of multiple transmitters and receivers and present the concept of radio transformer networks as a means to incorporate expert domain knolwedge in the machine learning model.

The application of convolutional neural networks on raw IQ samples for modulation classification are demonstrated to achieve competitive accuracy with respect to traditional schemes relying on expert features.

The open challenges and areas for future investigation are discussed.

Introduction

Communications is a field of rich expert knowledge about how to model channels of different types., compensate for various hardware imperfections, and design optimal signaling and detection schemes that ensure a reliable transfer of data.

As such, it is a complex and mature engineering field with many distinct areas of investigation which have all seen diminishing returns with regards to performance improvements, in particular on the physical layer. Because of this, there is a high bar of performance over which any machine learning (ML) or deep learning (DL) based approach must pass in order to provide tangible new benefits.

In domain such as computer vision and natural language processing, DL sines because it is difficult to characterize real world images or language with rigid mathematical methods. For example, while it is an almost impossible task to write a robust algorithm for detection of handwritten digits or objects in images, it is straightforward today to implement DL algorithms that learn to accomplish this task beyond human levels of accuracy.

In communications, however, we can design transmit signals that enable straightforward analytic algorithms for symbol detection for a variety of channel and system models (e.g. detection of a constellation symbol in additive Gaussian noise (AWGN). Thus, as long as such models sufficiently capture real effects, we do not expect DL to yield significant improvements on the physical layer.

Nevertheless, We believe that the DL applications which we explore in this paper are a useful insightful way of fundamentally rethinking the communications system design problem, and hold promise for performance improvements in complex communications scenarios that are difficult to describe with tractable mathematical models. The main contributions are as follows.

Learning full transmitter and receiver implementation for a given channel model which are optimized for a chosen loss function (e.g. minimizing block error rate (BLER)) is demonstrated. Such learned systems can be competitive with respect to the current state-of-the-art. The key idea is to represent transmitter, channel and receiver as one deep neural network which can be trained as an autoencoder. It can be applied to channel models and loss functions for which the optimal solutions are unknown.

The concept is extended to an adversarial network of multiple transmitter-receiver pairs competing for capacity. This leads to the interference channel for which finding the best signalling scheme is a long-standing research problem. Such a set up can also be represented as an NN with multiple inputs and outputs, and that all transmitter and receiver implementations can be jointly optimized with respect to a common or individual performance metrics.

Radio transformer networks (RTN) are introduced as a way to integrate expert knowledge into the DL model. RTN allows to carry predefined correction algorithms ("transformers") at the receiver (e.g., multiplication by a complex-valued number, convolution with a vector) which may be fed with parameters learned by another NN. This NN can be integrated into the end-to-end training prlcess of a task performed on the transformed signal (e.g., symbol detection).

The use of convolutional NNs on complex-valued IQ samples are studied for the problem of modulation classification.

A. Potential of DL for the Physical Layer

There are some reasons why DL could provide gains over existing physical layer algorithms.

First, most signal processing algorithms in communications have solid foundations in statistics and information theory and are often provably optimal for tractable mathematically models. These are generally linear, stationary, and have Gaussian statistics. A practical system, however, has many imperfections and non-linearities (e.g. non-linear power amplifier (PAs), finite resolution quantization) that can only be approximately captured by such models. For this reason, a DL-based communications system (or processing block) that does not require a mathematically tractable model and taht can be optimized for a specific hardware configuration and channel might be able to better optimize for such imperfections.

Second, one of the guiding principles of communications systems design is to split the signal processing into a chain of multiple independent blocks; each executing a well defined and isolated function (e.g., source/channel coding, modulation, channel estimation, equalization). Although this approach has led to the efficient, versatile, and controllable systems we have today, it is not clear that individually optimized processing blocks achieve the best possible end-to-end performance. For example, the separation of source and channel coding for many practical channels and short block lengths as well as separate coding and modulation are known to be sub-optimal. Attempts to jointly optimize each of these components, e.g., based on factor graphs, provide gains but lead to unwieldy and computationally complex systems. A learned end-to-end communications system, however, does not requires such a rigid modular structure as it is optimized for end-to-end performance.

Third, it has been shown that NNs are universal functioon approximators and rcent work has shown a remarkable capacity for algorithmic learning with recurrent NNs that are known to be Turing-complete.

Since the execution of NNs can be highly parallelized on concurrent architectures and easy implemented with low-precision data types, there is evidence that learned algorithms taking this form could be executed faster and at lower energy cost than their manually programmed counterparts.

Fourth, massively parallel processing architectures with distributed memory architectures, such as graphic processing units (GPUs) but also increasingly specialized chips for NN inference, have shown to be very energy efficient and capable of impressive computational throughput when fully utilized by concurrent algorithms. The performance of such architectures, however, has been largely limited by the ability of algorithms and higher level programming languages to make efficient use of them. The inherently concurrent nature of computation and memory access across wide and deep NNs has demonstrated a surprising ability to readily achieve application specific tuning or optimization required.

HIstorical context and related work

Applications of ML in communications have a long history covering a wide range of applications. These comprise channel modeling and prediction, localization, equalization, decoding, quantization, compression, demodulation, modulation recognition, and spectrum sensing to name a few.

However, to the best of our knowledge and due to the reasons mentioned above, few of these applications have been commonly adopted or led to a wide commercial success.