Библиографический список.
Разработка веб-приложения для масштабирования растровых изображений с использованием нейронных сетей

РефератПомощь в написанииУзнать стоимостьмоей работы

Требования к исходным кодам и языкам программирования Исходные коды программного продукта должны быть реализованы на языке C# для серверной части веб-приложения и на языке JavaScript для клиентской части. В качестве интегрированной среды разработки может использоваться любая среда разработки, с возможностью интеграции с MsBuild. В приоритете использование Visual Studio 2017 версии в качестве… Читать ещё >

Библиографический список. Разработка веб-приложения для масштабирования растровых изображений с использованием нейронных сетей (реферат, курсовая, диплом, контрольная)

1. Pu H.C., Lin C.T. A neural-network-based image resolution enhancement scheme for image resizing // International Journal of Image and Graphics. — 2002. — Т.2. — №.04. — С.503−517.
2. Krizhevsky A., Sutskever I., Hinton G.E. Imagenet classification with deep convolutional neural networks // Advances in neural information processing systems. — 2012. — С.1097−1105.
3. Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, и Gang Wang. Recent advances in convolutional neural networks. CoRR, 2015.
4. A.L. Samuel Some Studies in Machine Learning Using the Game of Checkers // IBM Journal. July 1959. P.210−229.
5. T.M. Mitchell Machine Learning. McGraw-Hill, 1997.
6. Воронцов К. В. Математические методы обучения по прецедентам (теория обучения машин) // Москва. — 2011.
7. Аверкин А. Н., Гаазе-Рапопорт М.Г., Поспелов Д. А. Толковый словарь по искусственному интеллекту. — Радио и связь, 1992.
8. Хайкин С. Нейронные сети: полный курс, 2-е издание. — Издательский дом Вильямс, 2008.
9. Круглов В. В., Дли М. И., Голунов Р. Ю. Нечеткая логика и искусственные нейронные сети. — М.: Физматлит, 2001.
10. Друки А. А. Применение свёрточных нейронных сетей для выделения и распознавания автомобильных номерных знаков на изображениях со сложным фоном // Известия ТПУ. 2014. № 5. URL: http://cyberleninka.ru/article/n/primenenie-svertochnyh-neyronnyh-setey-dlya-vydeleniya-i-raspoznavaniya-avtomobilnyh-nomernyh-znakov-na-izobrazheniyah-so-slozhnym (дата обращения: 11.03.2017).
11. Simard P.Y. et al. Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis // ICDAR. — 2003. — Т.3. — С.958−962.
12. Yann LeCun, J.S. Denker, S. Solla, R.E. Howard and L.D. Jackel: Optimal Brain Damage, in Touretzky, David (Eds), Advances in Neural Information Processing Systems 2 (NIPS*89), Morgan Kaufman, Denver, CO, 1990.
13. Hubel D.H., Wiesel T.N. Brain mechanisms of vision. — WH Freeman, 1979.
14. LeCun Y. et al. Gradient-based learning applied to document recognition // Proceedings of the IEEE. — 1998. — Т.86. — №.11. — С.2278−2324.
15. Lin M., Chen Q., Yan S. network in network // arXiv preprint arXiv: 1312.4400. — 2013.
16. Szegedy C. et al. Going deeper with convolutions // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. — 2015. — С.1−9.
17. CCIR I.656: Interfaces for Digital Component Video Signals in 525-Line and 625-Line Television Systems Operating at the 4: 2: 2 Level of Recommendation ITU-R BT.601. — 1998.
18. Glasner D., Bagon S., Irani M. Super-resolution from a single image // Computer Vision, 2009 IEEE 12th International Conference on. — IEEE, 2009. — С.349−356.
19. Dong C. et al. Image super-resolution using deep convolutional networks // IEEE transactions on pattern analysis and machine intelligence. — 2016. — Т.38. — №.2. — С.295−307.
20. Yang J. et al. Image super-resolution via sparse representation // IEEE transactions on image processing. — 2010. — Т. 19. — №.11. — С.2861−2873.
21. Johnson J., Alahi A., Fei-Fei L. Perceptual losses for real-time style transfer and super-resolution // European Conference on Computer Vision. — Springer International Publishing, 2016. — С.694−711.
22. Vinoski S. Advanced message queuing protocol // IEEE Internet Computing. — 2006. — Т.10. — №.6.

Приложение А. Листинг кода

using System;

using System. Collections. Generic;

using System. Linq;

using System. Threading. Tasks;

using ImageSuperResolution. SRCNN. Handler. Messages;

namespace ImageSuperResolution. SRCNN. Handler. Upscalling.

public class SRCNNHandler.

public SRCNNModelLayer [] ScaleModel { get; set; }.

public int Scale { get; set; } = 2;

private ImagePlane Normalize (ImageChannel image).

int width = image. Width;

int height = image. Height;

ImagePlane imagePlane = new ImagePlane (width, height);

if (imagePlane. Buffer. Length! = image. Buffer. Length).

throw new Exception («Assertion error: length»);

for (int i = 0; i < image. Buffer. Length; i++).

double temp = image. Buffer [i] / 255.0;

imagePlane. Buffer [i] = temp;

return imagePlane;

private ImageChannel Denormalize (ImagePlane imagePlane).

ImageChannel image = new ImageChannel (imagePlane. Width, imagePlane. Height);

for (int i = 0; i < imagePlane. Buffer. Length; i++).

double temp = Math. Round (imagePlane. Buffer [i] * 255, MidpointRounding. AwayFromZero);

image. Buffer [i] = temp > 255? (byte) 255: (byte) temp;

return image;

private ImagePlane [] Convolution (ImagePlane [] inputPlanes, double [] W, int nOutputPlane, double [] bias).

int width = inputPlanes [0]. Width;

int height = inputPlanes [0]. Height;

ImagePlane [] outputPlanes = new ImagePlane [nOutputPlane];

for (int o = 0; o < nOutputPlane; o++).

outputPlanes [o] = new ImagePlane (width — 2, height — 2);

double [] sumValues = new double [nOutputPlane];

double [] biasValues = new double [nOutputPlane];

bias. CopyTo (biasValues, 0);

for (int w = 1; w < width — 1; w++).

for (int h = 1; h < height — 1; h++).

biasValues. CopyTo (sumValues, 0); // leaky ReLU bias.

for (int i = 0; i < inputPlanes. Length; i++).

double i00 = inputPlanes [i]. GetValue (w — 1, h — 1);

double i10 = inputPlanes [i]. GetValue (w, h — 1);

double i20 = inputPlanes [i]. GetValue (w + 1, h — 1);

double i01 = inputPlanes [i]. GetValue (w — 1, h);

double i11 = inputPlanes [i]. GetValue (w, h);

double i21 = inputPlanes [i]. GetValue (w + 1, h);

double i02 = inputPlanes [i]. GetValue (w — 1, h + 1);

double i12 = inputPlanes [i]. GetValue (w, h + 1);

double i22 = inputPlanes [i]. GetValue (w + 1, h + 1);

for (int o = 0; o < nOutputPlane; o++).

int weightIndex = (o * inputPlanes. Length * 9) + (i * 9);

double value = sumValues [o];

value += i00 * W [weightIndex++];

value += i10 * W [weightIndex++];

value += i20 * W [weightIndex++];

value += i01 * W [weightIndex++];

value += i11 * W [weightIndex++];

value += i21 * W [weightIndex++];

value += i02 * W [weightIndex++];

value += i12 * W [weightIndex++];

value += i22 * W [weightIndex++];

sumValues [o] = value;

for (int o = 0; o < nOutputPlane; o++).

double v = sumValues [o];

if (v < 0).

v *= 0.1;

outputPlanes [o]. SetValue (w — 1, h — 1, v);

return outputPlanes;

private double [] [] GetWeights (SRCNNModelLayer [] model).

LinkedList arrayW = new LinkedList ();

foreach (SRCNNModelLayer layer in model).

arrayW. AddLast (layer. GetAllWeights ());

return arrayW. ToArray ();

private ImagePlane [] [] UpscaleBlocks (SRCNNModelLayer [] model, double [] [] weights, Action progressCallback, int blockGroup, int totalBlockGroups, params ImagePlane [] [] blocks).

List outputBlocks = new List ();

for (int b = 0; b < blocks. Length; b++).

ImagePlane [] inputBlock = blocks [b];

ImagePlane [] outputBlock = null;

for (int l = 0; l < model. Length; l++).

int nOutputPlane = model [l]. OutputPlanesCount;

// convolution.

outputBlock = Convolution (inputBlock, weights [l], nOutputPlane, model [l]. Bias);

inputBlock = outputBlock; // propagate output plane to next layer input.

// imageBlocks. Blocks [b] = null;

outputBlocks. Add (outputBlock);

int doneRatio = (int) Math. Round ((double) (100 * (b + 1)) / blocks. Length, MidpointRounding. AwayFromZero);

progressCallback (new BlockUpscalling (blockGroup, totalBlockGroups).

Percent = doneRatio.

});

// imageBlocks. Blocks = null;

return outputBlocks. ToArray ();

private ImageChannels UpscaleRgbAsync (ImageChannels channels, SRCNNModelLayer [] model, int scale, Action progressCallback, string phase).

ImagePlane [] inputPlanes = channels. ToArray (). Select ((image) =>

ImageChannel imgResized = scale == 1? image: image. Resize (scale);

// extrapolation for layer count (each convolution removes outer 1 pixel border).

ImageChannel imgExtra = imgResized. Extrapolation (model. Length);

return Normalize (imgExtra);

}). ToArray ();

// blocking.

ImageBlocks imageBlocks = ImagePlane. Blocking (inputPlanes);

inputPlanes = null;

// init W.

double [] [] weights = GetWeights (model);

var outputBlocks = imageBlocks. Blocks. AsParallel ().

. SelectMany ((block, index) => UpscaleBlocks (model, weights, progressCallback,.

index + 1, imageBlocks. Blocks. Count,.

block)).

. ToList ();

// de-blocking.

ImagePlane [] outputPlanes = ImagePlane. Deblocking (outputBlocks. ToArray (), imageBlocks. BlocksWidth, imageBlocks. BlocksHeight);

if (outputPlanes. Length! = 3).

throw new Exception («Output planes must be 3: color channel R, G, B. «);

ImageChannels outputChannels = new ImageChannels ().

Red = Denormalize (outputPlanes [0]),.

Green = Denormalize (outputPlanes [1]),.

Blue = Denormalize (outputPlanes [2]),.

Alpha = channels. Alpha.

};

return outputChannels;

public async Task UpscaleImage (byte [] image, int width, int height, Action done, Action progress).

TimeSpan timeSpanStart = TimeSpan. FromTicks (DateTime. Now. Ticks);

// decompose.

progress (new ProgressMessage (UpscallingStatuses. Decompose, «in process»));

ImageChannels channels = ImageChannel. ChannelDecompose (image, width, height);

progress (new ProgressMessage (UpscallingStatuses. Decompose, «ready»));

// calculate.

// Scaling all blocks.

ImageChannels upscaledChannels = await Task. run (() => UpscaleRgbAsync (channels, ScaleModel, Scale, progress, «scale»));

// Scaled all blocks.

// resize alpha channel.

ImageChannel imageA = Scale == 1? upscaledChannels. Alpha: upscaledChannels. Alpha. Resize (Scale);

if (imageA. Buffer. Length! = upscaledChannels. Red. Buffer. Length).

throw new Exception («A channel image size must be same with R channel image size»);

upscaledChannels. Alpha = imageA;

// recompose.

progress (new ProgressMessage (UpscallingStatuses.compose, «in process»));

byte [] upscaledImage = ImageChannel. ChannelCompose (upscaledChannels);

progress (new ProgressMessage (UpscallingStatuses.compose, «ready»));

TimeSpan elapsedTime = TimeSpan. FromTicks (DateTime. Now. Ticks) — timeSpanStart;

done (new ResultMessage (upscaledImage, upscaledChannels. Red. Width, upscaledChannels. Red. Height, elapsedTime));

public async void UpscaleImageAsync (byte [] image, int width, int height, Action done, Action progress).

await UpscaleImage (image, width, height, done, progress);

using System;

using System. Collections. Generic;

using System. Linq;

using System. Threading. Tasks;

namespace ImageSuperResolution. SRCNN. Handler. Upscalling.

public class ImageBlocks.

public List Blocks { get; set; }.

public int BlocksWidth { get; set; }.

public int BlocksHeight { get; set; }.

public ImageBlocks ().

Blocks = new List ();

using System;

using System. Collections. Generic;

using System. Linq;

using System. Threading. Tasks;

namespace ImageSuperResolution. SRCNN. Handler. Upscalling.

public class ImageChannel.

public int Width { get; set; }.

public int Height { get; set; }.

public byte [] Buffer { get; set; }.

public ImageChannel (int width, int height, byte [] buffer = null).

Width = width;

Height = height;

if (buffer! = null).

if (buffer. Length! = width * height).

throw new ArgumentException («Illegal buffer length»);

else.

Buffer = buffer;

else.

Buffer = new byte [width * height];

public static byte [] ChannelCompose (ImageChannels channels).

int width = channels. Red. Width;

int height = channels. Red. Height;

byte [] image = new byte [width * height * 4];

for (int i = 0; i < width * height; i++).

image [i * 4] = channels. Red. Buffer [i];

image [i * 4 + 1] = channels. Green. Buffer [i];

image [i * 4 + 2] = channels. Blue. Buffer [i];

image [i * 4 + 3] = channels. Alpha. Buffer [i];

return image;

public static ImageChannels ChannelDecompose (byte [] image, int width, int height).

ImageChannel imageR = new ImageChannel (width, height);

ImageChannel imageG = new ImageChannel (width, height);

ImageChannel imageB = new ImageChannel (width, height);

ImageChannel imageA = new ImageChannel (width, height);

for (int w = 0; w < width; w++).

for (int h = 0; h < height; h++).

int index = w + (h * width);

imageR. Buffer [index] = image [ (w * 4) + (h * width * 4)];

imageG. Buffer [index] = image [ (w * 4) + (h * width * 4) + 1];

imageB. Buffer [index] = image [ (w * 4) + (h * width * 4) + 2];

imageA. Buffer [index] = image [ (w * 4) + (h * width * 4) + 3];

return new ImageChannels ().

Alpha = imageA,.

Red = imageR,.

Green = imageG,.

Blue = imageB.

};

public byte [] ChannelDecompose (ImageChannels channels).

int width = channels. Red. Width;

int height = channels. Red. Height;

byte [] image = new byte [width * height * 4];

for (int i = 0; i < width * height; i++).

image [i * 4] = channels. Red. Buffer [i];

image [i * 4 + 1] = channels. Green. Buffer [i];

image [i * 4 + 2] = channels. Blue. Buffer [i];

image [i * 4 + 3] = channels. Alpha. Buffer [i];

return image;

private int ToIndex (int widthPosition, int heigthPosition, int width).

return widthPosition + heigthPosition * width;

public ImageChannel Extrapolation (int px).

int width = Width;

int height = Height;

ImageChannel extrapolatedImage = new ImageChannel (width + (2 * px), height + (2 * px));

for (int h = 0; h < height + (px * 2); h++).

for (int w = 0; w < width + (px * 2); w++).

int index = w + h * (width + (px * 2));

if (w < px).

// Left outer area.

if (h < px).

// Left upper area.

extrapolatedImage. Buffer [index] = Buffer [ToIndex (0, 0, width)];

else if (px + height <= h).

// Left lower area.

extrapolatedImage. Buffer [index] = Buffer [ToIndex (0, height — 1, width)];

else.

// Left outer area.

extrapolatedImage. Buffer [index] = Buffer [ToIndex (0, h — px, width)];

else if (px + width <= w).

// Right outer area.

if (h < px).

// Right upper area.

extrapolatedImage. Buffer [index] = Buffer [ToIndex (width — 1, 0, width)];

else if (px + height <= h).

// Right lower area.

extrapolatedImage. Buffer [index] = Buffer [ToIndex (width — 1, height — 1, width)];

else.

// Right outer area.

extrapolatedImage. Buffer [index] = Buffer [ToIndex (width — 1, h — px, width)];

else if (h < px).

// Upper outer area.

extrapolatedImage. Buffer [index] = Buffer [ToIndex (w — px, 0, width)];

else if (px + height <= h).

// Lower outer area.

extrapolatedImage. Buffer [index] = Buffer [ToIndex (w — px, height — 1, width)];

else.

// Inner area.

extrapolatedImage. Buffer [index] = Buffer [ToIndex (w — px, h — px, width)];

return extrapolatedImage;

public ImageChannel Resize (double scale).

int width = Width;

int height = Height;

int scaledWidth = (int) Math. Round (width * scale, MidpointRounding. AwayFromZero);

int scaledHeight = (int) Math. Round (height * scale, MidpointRounding. AwayFromZero);

ImageChannel scaledImage = new ImageChannel (scaledWidth, scaledHeight);

for (int w = 0; w < scaledWidth; w++).

for (int h = 0; h < scaledHeight; h++).

int scaledIndex = w + (h * scaledWidth);

int originalWidth = (int) Math. Round ((w + 1) / scale, MidpointRounding. AwayFromZero) — 1;

int originalHeight = (int) Math. Round ((h + 1) / scale, MidpointRounding. AwayFromZero) — 1;

int originalImageIndex = originalWidth + (originalHeight * width);

scaledImage. Buffer [scaledIndex] = Buffer [originalImageIndex];

return scaledImage;

using System;

using System. Collections. Generic;

using System. Linq;

using System. Threading. Tasks;

namespace ImageSuperResolution. SRCNN. Handler. Upscalling.

public class ImageChannels.

public ImageChannel Alpha { get; set; }.

public ImageChannel Red { get; set; }.

public ImageChannel Green { get; set; }.

public ImageChannel Blue { get; set; }.

public ImageChannel [] ToArray (bool ignoreAlpha = true).

if (ignoreAlpha).

return new ImageChannel [] { Red, Green, Blue };

else.

return new ImageChannel [] { Red, Green, Blue, Alpha};

using System;

using System. Collections. Generic;

using System. Linq;

using System. Threading. Tasks;

namespace ImageSuperResolution. SRCNN. Handler. Upscalling.

public class ImagePlane.

// Settings.

private const int BlockSize = 128;

private const int Overlap = 14;

// Props.

public int Width { get; set; }.

public int Height { get; set; }.

public double [] Buffer { get; private set; }.

public ImagePlane (int width, int height, double [] buffer = null).

Width = width;

Height = height;

if (buffer! = null).

if (buffer. Length! = width * height).

throw new ArgumentException («Illegal buffer length»);

else.

Buffer = buffer;

else.

Buffer = new double [width * height];

public long GetIndex (int widthPosition, int heightPosition).

return widthPosition + heightPosition * Width;

public double GetValue (int widthPosition, int heightPosition).

return Buffer [GetIndex (widthPosition, heightPosition)];

public void SetValue (int widthPosition, int heightPosition, double value).

Buffer [GetIndex (widthPosition, heightPosition)] = value;

public static ImageBlocks Blocking (ImagePlane [] initialPlanes).

int widthInput = initialPlanes [0]. Width;

int heightInput = initialPlanes [0]. Height;

int blocksWidth = (int) Math. Ceiling ((double) (widthInput — Overlap) / (BlockSize — Overlap));

int blocksHeight = (int) Math. Ceiling ((double) (heightInput — Overlap) / (BlockSize — Overlap));

int blocks = blocksWidth * blocksHeight;

ImageBlocks inputBlocks = new ImageBlocks () // [[block0_R, block0_G, block0_B], [block1_R,.]. ].

BlocksWidth = blocksWidth,.

BlocksHeight = blocksHeight.

};

for (int b = 0; b < blocks; b++).

int blockWidthIndex = b % blocksWidth;

int blockHeightIndex = (int) Math. Floor ((double) (b / blocksWidth));

int blockWidth;

int blockHeight;

if (blockWidthIndex == blocksWidth — 1).

blockWidth = widthInput — ((BlockSize — Overlap) * blockWidthIndex); // right end block.

else.

blockWidth = BlockSize;

if (blockHeightIndex == blocksHeight — 1).

blockHeight = heightInput — ((BlockSize — Overlap) * blockHeightIndex); // bottom end block.

else.

blockHeight = BlockSize;

ImagePlane [] channels = new ImagePlane [initialPlanes. Length];

for (int i = 0; i < channels. Length; i++).

channels [i] = new ImagePlane (blockWidth, blockHeight);

for (int w = 0; w < blockWidth; w++).

for (int h = 0; h < blockHeight; h++).

for (int n = 0; n < initialPlanes. Length; n++).

int targetIndexW = blockWidthIndex * (BlockSize — Overlap) + w;

int targetIndexH = blockHeightIndex * (BlockSize — Overlap) + h;

ImagePlane channel = initialPlanes [n];

double channelValue = channel. GetValue (targetIndexW, targetIndexH);

channels [n]. SetValue (w, h, channelValue);

inputBlocks. Blocks. Add (channels);

return inputBlocks;

public static ImagePlane [] Deblocking (ImagePlane [] [] outputBlocks, int blocksWidth, int blocksHeight).

int blockSize = outputBlocks [0] [0]. Width;

int width = 0;

for (int b = 0; b < blocksWidth; b++).

width += outputBlocks [b] [0]. Width;

int height = 0;

for (int b = 0; b < blocksWidth * blocksHeight; b += blocksWidth).

height += outputBlocks [b] [0]. Height;

// console. log («result image width: «+ width + «height: «+ height);

List outputPlanes = new List (); // [planeR, planeG, planeB].

for (int b = 0; b < outputBlocks. Length; b++).

ImagePlane [] block = outputBlocks [b];

int blockWidthIndex = b % blocksWidth;

int blockHeightIndex = (int) Math. Floor ((double) b / blocksWidth);

for (int n = 0; n < block. Length; n++).

if (outputPlanes. Count < n + 1).

outputPlanes. Add (new ImagePlane (width, height));

else if (outputPlanes [n] == null).

outputPlanes [n] = new ImagePlane (width, height);

ImagePlane channelBlock = block [n];

for (int w = 0; w < channelBlock. Width; w++).

for (int h = 0; h < channelBlock. Height; h++).

int targetIndexW = blockWidthIndex * blockSize + w;

int targetIndexH = blockHeightIndex * blockSize + h;

int targetIndex = targetIndexH * width + targetIndexW;

double channelValue = channelBlock. GetValue (w, h);

outputPlanes [n]. Buffer [targetIndex] = channelValue;

return outputPlanes. ToArray ();

using System;

using System. Collections. Generic;

using System. IO;

using System. Linq;

using System. Threading. Tasks;

using Newtonsoft. Json;

using Newtonsoft. Json. Linq;

namespace ImageSuperResolution. SRCNN. Handler. Upscalling.

public class SRCNNModelLayer.

public class ModelConfig.

public string ModelName { get; set; }.

public int ScaleFactor { get; set; }.

public int Channels { get; set; }.

public int Offset { get; set; }.

[JsonProperty («weight»)].

public double [] [] [] [] Weights { get; internal set; }.

public int StrideHeight { get; set; }.

public int StrideWidth { get; set; }.

[JsonProperty («nInputPlane»)].

public int InputPlanesCount { get; set; }.

[JsonProperty («nOutputPlane»)].

public int OutputPlanesCount { get; set; }.

public int KernelWidth { get; set; }.

public int KernelHeight { get; set; }.

public ModelConfig ModelInfo { get; set; }.

[JsonProperty («bias»)].

public double [] Bias { get; set; }.

public SRCNNModelLayer ().

public double [] GetAllWeights ().

LinkedList allWeights = new LinkedList ();

foreach (double [] [] [] weightForOutputPlane in Weights).

foreach (double [] [] weightMatrix in weightForOutputPlane).

foreach (double [] weightVector in weightMatrix).

foreach (double omgForGodsSakeFinalWeight in weightVector).

allWeights. AddLast (omgForGodsSakeFinalWeight);

return allWeights. ToArray ();

public static SRCNNModelLayer [] ReadModel (string json).

JArray jsonModel = JArray. Parse (json);

List layers = new List ();

foreach (JToken jsonLayer in jsonModel).

var layer = JsonConvert. DeserializeObject (jsonLayer. ToString ());

layers. Add (layer);

return layers. ToArray ();

using System;

using System. Collections. Generic;

using System. Drawing;

using System. IO;

using System. Threading;

using System. Threading. Tasks;

using ImageSuperResolution. SRCNN. Handler. Messages;

using ImageSuperResolution. SRCNN. Handler. Upscalling;

using RabbitMQ. Client;

using RabbitMQ. Client. Events;

namespace ImageSuperResolution. SRCNN. Handler. Services.

public class QueueHandler: UpscallingServiceBase.

private readonly string _queueInput;

private readonly string _queueOutput;

private string _imageConsumerTag;

private IConnection _connectionToMQ;

private IModel _channelInput;

private IModel _channelOutput;

private readonly List _imagePrecessingTasks;

private readonly CancellationTokenSource _cancellationTokenSource;

public QueueHandler (): base ().

_queueInput = «image_input» ;

_queueOutput = «image_output» ;

public override void Start ().

ConnectToMq ();

public override void Stop ().

_channelInput. BasicCancel (_imageConsumerTag);

_connectionToMQ. Close ();

private void ConnectToMq ().

var factory = new ConnectionFactory ().

HostName = «localhost» .

};

_connectionToMQ = factory. CreateConnection ();

_channelInput = _connectionToMQ. CreateModel ();

_channelInput. QueueDeclare (.

queue: _queueInput,.

durable: false,.

exclusive: false,.

autoDelete: false,.

arguments: null);

_channelInput. BasicQos (0, 1, false);

var imageConsumer = new EventingBasicConsumer (_channelInput);

imageConsumer. Received += MessageReceived;

_imageConsumerTag = _channelInput. BasicConsume (_queueInput, false, imageConsumer);

private void InitConsumer (IModel channelInput).

channelInput. QueueDeclare (.

queue: _queueInput,.

durable: false,.

exclusive: false,.

autoDelete: false,.

arguments: null);

channelInput. BasicQos (0, 1, false);

var imageConsumer = new EventingBasicConsumer (channelInput);

imageConsumer. Received += MessageReceived;

_imageConsumerTag = channelInput. BasicConsume (_queueInput, false, imageConsumer);

private void PublishMessage (IModel outputChannel, Guid ticket, byte [] messageBody).

var props = outputChannel. CreateBasicProperties ();

props. CorrelationId = ticket. ToString ();

outputChannel. BasicPublish (exchange: «», routingKey: _queueOutput,.

basicProperties: props, body: messageBody);

private void MessageReceived (object sender, BasicDeliverEventArgs e).

byte [] imageData = e. Body;

if (! _cancellationTokenSource. IsCancellationRequested).

using (var originalImage = new Bitmap (GetImageFromByteArray (imageData))).

byte [] rgba = ImageUtils. GetRgbaFromBitmap (originalImage);

SRCNNHandler srcnn = new SRCNNHandler ().

Scale = 2, ScaleModel = Model.

};

int width = originalImage. Width;

int height = originalImage. Height;

Task upscallingTask = Task. run (.

() => srcnn. UpscaleImageAsync (rgba, width, height, ResultHandling,.

ProgressLogging), _cancellationTokenSource. Token);

_imagePrecessingTasks. RemoveAll (t => t. IsCompleted);

_imagePrecessingTasks. Add (upscallingTask);

private void ResultHandling (ResultMessage result).

throw new NotImplementedException ();

private void ProgressLogging (ProgressMessage message).

Console. WriteLine (message);

private Image GetImageFromByteArray (byte [] file).

using (MemoryStream mStream = new MemoryStream (file)).

return Image. FromStream (mStream);

using System. IO;

using System. Linq;

using System. Reflection;

using ImageSuperResolution. SRCNN. Handler. Upscalling;

namespace ImageSuperResolution. SRCNN. Handler. Services.

public abstract class UpscallingServiceBase.

protected readonly SRCNNModelLayer [] Model;

public UpscallingServiceBase ().

Model = LoadModel ();

public abstract void Start ();

public abstract void Stop ();

private SRCNNModelLayer [] LoadModel ().

var assembly = typeof (UpscallingServiceBase). GetTypeInfo (). Assembly;

string [] names = assembly. GetManifestResourceNames ();

Stream modelResource = assembly. GetManifestResourceStream (names. First (name => name. Contains («model. json»)));

string jsonModel;

using (var sr = new StreamReader (modelResource)).

jsonModel = sr. ReadToEnd ();

return SRCNNModelLayer. ReadModel (jsonModel);

using System;

using System. componentModel;

using System. Reflection;

namespace ImageSuperResolution. SRCNN. Handler. Messages. Helpers.

public static class ReflectionHelpers.

public static string GetCustomDescription (object objEnum).

var fi = objEnum. GetType (). GetField (objEnum. ToString ());

var attributes = (DescriptionAttribute []) fi. GetCustomAttributes (typeof (DescriptionAttribute), false);

return (attributes. Length > 0)? attributes [0]. Description: objEnum. ToString ();

public static string Description (this Enum value).

return GetCustomDescription (value);

using System;

using System. Collections. Generic;

using System. Drawing;

using System. Linq;

using System. Threading. Tasks;

namespace ImageSuperResolution. SRCNN. Handler. Messages.

public class BlockUpscalling: ProgressMessage.

public int TotalBlocks { get; set; }.

public int BlockNumber { get; set; }.

public BlockUpscalling (int blockNumber, int totalBlocks): base (UpscallingStatuses. UpscallingBlock, «scalling block»).

BlockNumber = blockNumber;

TotalBlocks = totalBlocks;

public override string ToString ().

return $" {base. ToString () } (block {BlockNumber} of {TotalBlocks})" ;

using System;

using System. Collections. Generic;

using System. Linq;

using System. Threading. Tasks;

using ImageSuperResolution. SRCNN. Handler. Messages. Helpers;

namespace ImageSuperResolution. SRCNN. Handler. Messages.

public class ProgressMessage.

public int? Percent { get; set; } = null;

public string Message { get; set; }.

public UpscallingStatuses Phase { get; set; }.

public ProgressMessage (UpscallingStatuses phase, string message).

Phase = phase;

Message = message;

public ProgressMessage ().

public override string ToString ().

if (Percent! = null).

return $" {Phase. Description () } ({Percent}%): {Message}" ;

else.

return $" {Phase. Description () }: {Message}" ;

using System;

using System. Collections. Generic;

using System. Linq;

using System. Threading. Tasks;

namespace ImageSuperResolution. SRCNN. Handler. Messages.

public class ResultMessage.

public byte [] ImageRgba { get; set; }.

public TimeSpan ElapsedTime { get; set; }.

public int ImageWidth { get; set; }.

public int ImageHeight { get; set; }.

public ResultMessage (byte [] image, int width, int height, TimeSpan time).

ImageRgba = image;

ImageWidth = width;

ImageHeight = height;

ElapsedTime = time;

public override string ToString ().

return $" Image ({ImageRgba. Length} bytes) was upscaled to width {ImageWidth}px and height {ImageHeight}px for {ElapsedTime: g}" ;

using System;

using System. Collections. Generic;

using System. componentModel;

using System. Linq;

using System. Threading. Tasks;

namespace ImageSuperResolution. SRCNN. Handler. Messages.

public enum UpscallingStatuses.

[Description («Decomposing»)].

Decompose,.

[Description («Upscalling block»)].

UpscallingBlock,.

[Description («Composing»)].

Compose.

using System;

using System. Collections. Generic;

using System. Drawing;

using System. Drawing. Imaging;

using System. IO;

using System. Threading;

using System. Threading. Tasks;

using ImageSuperResolution. SRCNN. Handler. Messages;

using ImageSuperResolution. SRCNN. Handler. Upscalling;

namespace ImageSuperResolution. SRCNN. Handler. Services.

public class FileStubHandler: UpscallingServiceBase.

private string _directoryInput;

private string _directoryOutput;

private ImageWatcherService _watcher;

private readonly List _imagePrecessingTasks;

private readonly CancellationTokenSource _cancellationTokenSource;

public FileStubHandler (): base ().

InitFolders ();

_imagePrecessingTasks = new List ();

_cancellationTokenSource = new CancellationTokenSource ();

public override void Start ().

_watcher = new ImageWatcherService (_directoryInput);

_watcher. OnNewFileFound += ProceedImage;

_watcher. StartWatching ();

public override void Stop ().

Console. WriteLine («Cancel requested. «);

_watcher. StopWatching ();

_cancellationTokenSource. Cancel ();

Task. WaitAll (_imagePrecessingTasks. ToArray ());

private void ProceedImage (object sender, string pathToImage).

if (! _cancellationTokenSource. IsCancellationRequested).

using (var originalImage = new Bitmap (pathToImage)).

var rgba = ImageUtils. GetRgbaFromBitmap (originalImage);

SRCNNHandler srcnn = new SRCNNHandler ().

Scale = 2, ScaleModel = Model.

};

int width = originalImage. Width;

int height = originalImage. Height;

Task upscallingTask = Task. run (.

() => srcnn. UpscaleImageAsync (rgba, width, height, ResultHandling,.

ProgressLogging), _cancellationTokenSource. Token);

_imagePrecessingTasks. RemoveAll (t => t. IsCompleted);

_imagePrecessingTasks. Add (upscallingTask);

File. Delete (pathToImage);

private void ProgressLogging (ProgressMessage message).

Console. WriteLine (message);

private void ResultHandling (ResultMessage result).

var newImage = ImageUtils. GetBitmapFromRgba (result. ImageWidth, result. ImageHeight, result. ImageRgba);

string filename = $" {DateTime. Now: yy-MM-dd hh-mm-ss}. png" ;

newImage. Save (Path.combine (_directoryOutput, filename), ImageFormat. Png);

Console. WriteLine (result);

private void InitFolders ().

_directoryInput = Path. combine (Directory. GetCurrentDirectory (), «ImageInputStub»);

_directoryOutput = Path. combine (Directory. GetCurrentDirectory (), «ImageOutputStub»);

Directory. CreateDirectory (_directoryInput);

Directory. CreateDirectory (_directoryOutput);

using System;

using System. Collections. Generic;

using System. IO;

using System. Linq;

using System. Threading;

using System. Threading. Tasks;

namespace ImageSuperResolution. SRCNN. Handler. Services.

public class ImageWatcherService.

public string DirectoryWithImages { get; private set; }.

private CancellationTokenSource _cancellationTokenSource;

public event EventHandler OnNewFileFound;

public ImageWatcherService (string directoryWithImages).

DirectoryWithImages = directoryWithImages;

public void StartWatching ().

_cancellationTokenSource = new CancellationTokenSource ();

Watching (_cancellationTokenSource. Token);

public void StopWatching ().

_cancellationTokenSource. Cancel ();

private void Watching (CancellationToken token).

Task.run (() =>

while (true).

token. ThrowIfCancellationRequested ();

string [] imagePaths = Directory. GetFiles (DirectoryWithImages);

if (imagePaths. Any ()).

string imagePath = imagePaths. First (s => s. EndsWith («. bmp») || s. EndsWith («. png») || s. EndsWith («. jpg»));

OnNewFileFound?. Invoke (this, imagePath);

}, token);

Приложение B. Диаграмма последовательностей загрузки изображения для масштабирования

Приложение С. Диаграмма классов модуля масштабирования.

Приложение D. Техническое задание

НЕЙРОСЕТЕВОЕ ВЕБ_ПРИЛОЖЕНИЕ ДЛЯ УВЕЛИЧЕНИЯ РАСТРОВЫХ ИЗОБРАЖЕНИЙ.

Техническое задание.

ЛИСТ УТВЕРЖДЕНИЯ

Введение

Нейросетевое веб-приложение для увеличения растровых изображений (Neural network based web application for upscalling bitmap images). Программа предназначена для использования разработчиками игр, графических редакторов, дизайнерами, журналистами и фотографами.

Назначение разработки Функциональным назначением программы является предоставление пользователю возможности увеличения размеров изображения в два раза с минимальной потерей в качестве при помощи свёрточной нейронной сети.

Требования к программе или программному изделию.

1.1 Требования к функциональным характеристикам Требования к составу выполняемых функций.

— Загрузка растровых изображений.
— Увеличение изображений с минимальной потерей в качестве.
— Возможность получить результат как на открытой веб-странице, так и по прямой ссылке.
1.2 Требования к надежности

Требования к обеспечению надежного (устойчивого) функционирования программы Надежное (устойчивое) функционирование программы должно быть обеспечено выполнением заказчиком совокупности организационно-технических мероприятий, перечень которых приведен ниже:

— организацией бесперебойного питания технических средств;
— использованием лицензионного программного обеспечения;
— регулярным выполнением рекомендаций Министерства труда и социального развития РФ, изложенных в Постановлении от 23 июля 1998 г. «Об утверждении межотраслевых типовых норм времени на работы по сервисному обслуживанию ПЭВМ и оргтехники и сопровождению программных средств» ;
— регулярным выполнением требований ГОСТ 51 188–98. Защита инфоpмации. Испытания пpогpаммных сpедств на наличие компьютеpных виpусов.
1.3 Условия эксплуатации

Климатические условия эксплуатации Климатические условия эксплуатации, при которых должны обеспечиваться заданные характеристики, должны удовлетворять требованиям, предъявляемым к техническим средствам в части условий их эксплуатации.

1.4 Требования к составу и параметрам технических средств Требования для сервера.

1) Процессор Intel Xeon E5−2667 Haswell с тактовой частотой 3,2 ГГц, может быть выделено 4 и более ядер.
2) Оперативная память 8 Гб, не менее.
3) Физическая память 500 Гб, не менее.

Требования для клиента.

1) У пользователя на персональном компьютере должен быть установлен браузер, рекомендуется Chrome версии 55 и выше.
1.5 Требования к информационной и программной совместимости

Требования к информационным структурам и методам решения Требования к информационным структурам не предъявляются.

Требования к защите информации и программ Требования к защите информации и программ не предъявляются.

1.6 Требования к маркировке и упаковке.

Маркировка и упаковка не требуются.

1.7 Требования к транспортированию и хранению.

Транспортирование и хранение не требуются.

1.8 Специальные требования
1) Клиентская часть должна быть спроектирована в соответствии с правилами User Experience Design.
2) Серверная часть должна быть спроектирована и разработана в соответствии с общепринятыми шаблонами проектирования (software pattern), если это необходимо и возможно в конкретной ситуации.
3) Приложение должно корректно работать в браузере Chrome версии 55 и выше.

Стадии и этапы разработки.

1.9 Стадии разработки
1) Техническое задание.
2) Технический проект.
3) Внедрение.
1.10 Этапы разработки

На стадии «Техническое задание» должен быть выполнен этап разработки, согласования и утверждения настоящего технического задания.

На стадии «Технический проект» должны быть выполнены следующие этапы работ:

1) Проектирование архитектуры нейронной сети.
2) Проектирование архитектуры модуля масштабирования.
3) Проектирование архитектуры веб-приложения.
4) Реализация модуля масштабирования.
5) Реализация каналов обмена сообщениями между узлами приложения.
6) Реализация графического интерфейса.
7) Реализация серверной части веб-приложения.
8) Настройка брокера очередей сообщений.
9) Создание Docker-контейнеров для автоматического развертывания на серверах.

Abstract.

This final qualifying work contains an analysis of the method of obtaining super-resolution using a convolutional neural network (SRCNN), as well as a description of an application designed to demonstrate the operation of the selected neural network. Graduation qualification work occupies 38 pages of A4 format, includes 14 figures, 1 table, 4 annexes and consists of 3 chapters: Analysis of the subject area and methods of machine learning, Design of neural network architecture and Application development.

The first chapter analyzes the subject area of?? machine learning and describes the operation principle of a convolutional neural network. The second chapter describes the architecture of the SRCNN convolutional neural network, and its modification is designed. The third chapter describes the implementation of two versions of the application for demonstrating the operation of a neural network. The aim of this work is the development of a neural network module for upscaling images with minimal loss in quality, as well as demonstrating the use of this module through a web interface.

Показать весь текст

Заполнить форму текущей работой