基于神经网络的通信系统建模与仿真

论文总字数：67087字

摘 要

神经网络是模仿人脑神经网络建成的。它通过对人脑神经网络的抽象和建模，使计算机拥有像人一样的学习能力。神经网络通常由很多隐藏层组成。这不仅可以降低网络所需要的参数数目，也可以增强网络表示非线性函数的能力。神经网络的损失函数默认为模型分布与经验分布之间的交叉熵，最大似然准则为我们在不同模型下均提供了损失函数的较好选择。神经网络的发展一度处于低谷，然而反向传播算法的提出让网络能以廉价的代价进行梯度更新，从而推进学习过程。目前，神经网络除了在图像处理、语音识别、人脸识别、自然语言处理等各个领域有诸多应用以外，它在数字通信中的发展也得到了广泛的关注。通信的基本问题是在接收端再生发送端发送的消息。然而，由于实际应用的复杂性，研究者们通常将整个通信系统划分成不同模块，从而分别设计和优化，因此现在神经网络在数字通信系统中的应用大多针对模块，如信号检测、信号分类、信道估计、非线性信道的建模与识别、信道均衡、编解码、量化、非线性滤波和频谱扩展等等，还很少有研究者将神经网络应用于端到端的通信系统设计。而基于神经网络的端到端通信系统要求设计者将整个系统视为一体，无论是数据的前向传播代价计算，还是反向传播参数更新，都从整体的观念加以考量。自编码器是一种在输出端再生输入的神经网络。它与通信系统的基本问题不谋而合，因可以应用在解决通信系统的基本问题上。但是，当输入的维数增加时，自编码器又面临着网络参数巨增的问题。本论文基于传统自编码器，提出了单用户与多用户场景下的短码通信架构。该架构在传统自编码器的基础上，引入一位有效编码，从而将重构输入这一目标转化为样本分类问题，因而能利用交叉熵损失作为代价函数来最小化模型之间的分布差异，准确率高。多用户通信场景沿用的是单用户通信的架构，而在计算损失函数的时候引入新的具有自适应性的超参数，来获得多用户通信下的最优性能。仿真结果表明，基于自编码器的通信系统与传统通信系统相比，具有一定的性能优势。另外，通信系统中的噪声对神经网络来说是一种数据增强，可以有效防止网络过拟合。最后，本论文针对长码问题，还提出了一种新颖的卷积自编码器架构。它有效地利用了卷积神经网络稀疏交互和参数共享的特点，来降低网络所需的参数数目，从而应对输入维数过高的情形。

关键词：神经网络，自编码器，通信系统，端到端

Modeling and Simulation of Communication System

based on Neural Network

04014444 Runing Xu

Advisor Wei Xu

Abstract

Neural network(NN) is built according to human’s brain. It could have the ability to learn as human when fundamental things are abstracted from the network in our brain. Ordinarily, neural network is made of many layers, which are called hidden layers. These layers can not only reduce the parameters that the NN has, but also reinforce the non-linear feature of the network. The default loss function is the cross entropy of the distribution that the model output and that of the dataset. Maximum likelihood criterion offers a nice choice in spite of different models. Backpropagation algorithm can help gradient to update more hardware-efficiently, thus reviving the development of NN. Recently, in addition to image processing, speech recognition, face recognition and natural language processing, BP algorithm has been received great attention in the area of digital communication. It is a fundamental question in Communication that our task is to reconstruct the transmitted signal after it is received. Due to the complex situation, researches usually divide the whole system into several modules, including channel estimation, channel recognition, channel equation, signal classification, signal detection, coding and decoding, quantization, spectrum spread and non-linear filter, which can be optimized individually. Therefore, most modern applications of NN in digital communication are designed for modules, not the whole system. End-to-end communication system based on NNs requires designers to view the system as a whole, no matter whether the data is transmitted forward to calculate loss or backward to update gradients. Autoencoder is one of the NNs which needs to reconstruct the input at the receiver. Its operation and function is highly relative to the fundamental question of Communication. So it can be used to solve it. But when the size of input increases, it confronts the problem of tremendous parameters. This paper proposes an architecture based on autoencoder to accomplish the task of communication of single user and multiple users. In this architecture, reconstruction, the goal of traditional autoencoders, is transformed into classification since one-hot vector is used. Loss calculation can be easily achieved by cross entropy. In multiple users case, a new hyperparameter is introduced to decide the rate of loss that each user contributes and it is adaptive to achieve the best performance. Noise, which is annoying in Communication, is actually a method to reinforce the dataset to prevent the NN from overfitting. Finally, for the problem of high-dimension input, a novel convolutional autoencoder which has the advantage of sparse interaction and share parameters, is proposed to reduce the parameters of NN.

KEY WORDS: nerual network, autoencoder, communication system, end-to-end

目 录

摘要 I

Abstract 1

第一章 绪论 1

1.1 本文研究背景 1

1.1.1 神经网络的发展及研究现状 1

1.1.2 神经网络在数字通信中的应用 2

1.1.3 基于神经网络的端到端通信系统研究现状 9

1.2 本文研究内容 11

第二章 神经网络基础 13

2.1 引言 13

2.2 机器学习基础 13

2.2.1 概述 13

2.2.2 容量、过拟合和欠拟合 14

2.2.3 最大似然估计 16

2.2.4 梯度下降 16

2.3 深度前馈网络 17

2.3.1 线性模型 18

2.3.2 非线性模型 19

2.3.3 隐藏单元 19

2.3.4 输出单元 21

2.3.5 梯度下降及反向传播 22

2.3.6 万能近似定理和深度 23

2.4 卷积神经网络 23

2.4.1 稀疏交互 23

2.4.2 参数共享 24

2.4.3 池化 24

2.4.4 架构 25

2.5 学习和优化策略 26

2.5.1 正则化 26

2.5.2 数据集增强 26

2.5.3 提前终止 27

第三章 基于自编码器的端到端通信系统 28

3.1 引言 28

3.2 自编码器 28

3.3 端到端单用户通信 29

3.3.1 问题描述 29

3.3.2 架构搭建 30

3.3.3 仿真结果与分析 31

3.4 端到端多用户通信 42

3.4.1 问题描述 42

3.4.2 架构搭建 43

3.4.3 仿真结果与分析 43

第四章 基于卷积自编码器的端到端通信系统 45

4.1 引言 45

4.2 卷积自编码器 45

4.3 端到端单用户通信 46

4.3.1 问题描述 46

4.3.2 架构搭建 46

第五章 总结 48

5.1 论文主要贡献 48

5.2 进一步研究的方向 49

致谢 50

剩余内容已隐藏，请支付后下载全文，论文总字数：67087字