blog Archives - Page 8 of 9 - Program Studi S1 Informatika

Federated optimization in heterogeneous networks

January 5, 2023/in Publikasi /by Program Studi Informatika

Federated optimization in heterogeneous networks

by Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, Virginia Smith

Federated Learning is a distributed learning paradigm with two key challenges that differentiate it from traditional distributed optimization: (1) significant variability in terms of the systems characteristics on each device in the network (systems heterogeneity), and (2) non-identically distributed data across the network (statistical heterogeneity). In this work, we introduce a framework, FedProx, to tackle heterogeneity in federated networks. FedProx can be viewed as a generalization and re-parametrization of FedAvg, the current state-of-the-art method for federated learning. While this re-parameterization makes only minor modifications to the method itself, these modifications have important ramifications both in theory and in practice. Theoretically, we provide convergence guarantees for our framework when learning over data from non-identical distributions (statistical heterogeneity), and while adhering to device-level systems constraints by allowing each participating device to perform a variable amount of work (systems heterogeneity). Practically, we demonstrate that FedProx allows for more robust convergence than FedAvg across a suite of realistic federated datasets. In particular, in highly heterogeneous settings, FedProx demonstrates significantly more stable and accurate convergence behavior relative to FedAvg—improving absolute test accuracy by 18.8% on average.

Graph convolutional neural networks for web-scale recommender systems

January 5, 2023/in Publikasi /by Program Studi Informatika

Graph convolutional neural networks for web-scale recommender systems

by Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton

ACM Digital Library

Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains an unsolved challenge. Here we describe a large-scale deep recommendation engine that we developed and deployed at Pinterest. We develop a data-efficient Graph Convolutional Network (GCN) algorithm, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information. Compared to prior GCN approaches, we develop a novel method based on highly efficient random walks to structure the convolutions and design a novel training strategy that relies on harder-and-harder training examples to improve robustness and convergence of the model. We also develop an efficient MapReduce model inference algorithm to generate embeddings using a trained model. Overall, we can train on and embed graphs that are four orders of magnitude larger than typical GCN implementations. We show how GCN embeddings can be used to make high-quality recommendations in various settings at Pinterest, which has a massive underlying graph with 3 billion nodes representing pins and boards, and 17 billion edges. According to offline metrics, user studies, as well as A/B tests, our approach generates higher-quality recommendations than comparable deep learning based systems. To our knowledge, this is by far the largest application of deep graph embeddings to date and paves the way for a new generation of web-scale recommender systems based on graph convolutional architectures.

Convolutional neural networks: an overview and application in radiology

January 5, 2023/in Publikasi /by Program Studi Informatika

Convolutional neural networks: an overview and application in radiology

by R Yamashita, M Nishio, RKG Do, K Togashi

Insights into imaging

Convolutional neural network (CNN), a class of artificial neural networks that has become dominant in various computer vision tasks, is attracting interest across a variety of domains, including radiology. CNN is designed to automatically and adaptively learn spatial hierarchies of features through backpropagation by using multiple building blocks, such as convolution layers, pooling layers, and fully connected layers. This review article offers a perspective on the basic concepts of CNN and its application to various radiological tasks, and discusses its challenges and future directions in the field of radiology. Two challenges in applying CNN to radiological tasks, small dataset and overfitting, will also be covered in this article, as well as techniques to minimize them. Being familiar with the concepts and advantages, as well as limitations, of CNN is essential to leverage its potential in diagnostic radiology, with the goal of augmenting the performance of radiologists and improving patient care.

Key Points

• Convolutional neural network is a class of deep learning methods which has become dominant in various computer vision tasks and is attracting interest across a variety of domains, including radiology.

• Convolutional neural network is composed of multiple building blocks, such as convolution layers, pooling layers, and fully connected layers, and is designed to automatically and adaptively learn spatial hierarchies of features through a backpropagation algorithm.

• Familiarity with the concepts and advantages, as well as limitations, of convolutional neural network is essential to leverage its potential to improve radiologist performance and, eventually, patient care.

The unreasonable effectiveness of deep features as a perceptual metric

January 5, 2023/in Publikasi /by Program Studi Informatika

The unreasonable effectiveness of deep features as a perceptual metric

by Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, Oliver Wang

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions, and fail to account for many nuances of human perception. Recently, the deep learning community has found that features of the VGG network trained on ImageNet classification has been remarkably useful as a training loss for image synthesis. But how perceptual are these so-called “perceptual losses”? What elements are critical for their success? To answer these questions, we introduce a new dataset of human perceptual similarity judgments. We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics. We find that deep features outperform all previous metrics by large margins on our dataset. More surprisingly, this result is not restricted to ImageNet-trained VGG features, but holds across different deep architectures and levels of supervision (supervised, self-supervised, or even unsupervised). Our results suggest that perceptual similarity is an emergent property shared across deep visual representations.

A comprehensive survey on graph neural networks

January 5, 2023/in Publikasi /by Program Studi Informatika

A comprehensive survey on graph neural networks

by Zonghan Wu; Shirui Pan; Fengwen Chen; Guodong Long; Chengqi Zhang

IEEE Transactions on Neural Networks and Learning Systems

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications, where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on the existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this article, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art GNNs into four categories, namely, recurrent GNNs, convolutional GNNs, graph autoencoders, and spatial-temporal GNNs. We further discuss the applications of GNNs across various domains and summarize the open-source codes, benchmark data sets, and model evaluation of GNNs. Finally, we propose potential research directions in this rapidly growing field.

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

January 5, 2023/in Publikasi /by Program Studi Informatika

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

by M Raissi, P Perdikaris, GE Karniadakis

Journal of Computational physics

We introduce physics-informed neural networks – neural networks that are trained to solve supervised learning tasks while respecting any given laws of physics described by general nonlinear partial differential equations. In this work, we present our developments in the context of solving two main classes of problems: data-driven solution and data-driven discovery of partial differential equations. Depending on the nature and arrangement of the available data, we devise two distinct types of algorithms, namely continuous time and discrete time models. The first type of models forms a new family of data-efficient spatio-temporal function approximators, while the latter type allows the use of arbitrarily accurate implicit Runge–Kutta time stepping schemes with unlimited number of stages. The effectiveness of the proposed framework is demonstrated through a collection of classical problems in fluids, quantum mechanics, reaction–diffusion systems, and the propagation of nonlinear shallow-water waves.

Achieving a Covert Channel over an Open Blockchain Network

January 4, 2023/in Publikasi /by Informatika

Achieving a Covert Channel over an Open Blockchain Network

by Feng Gao, Liehuang Zhu, Keke Gai, Can Zhang, Sheng Liu

Institute of Electrical and Electronics Engineers (IEEE)

Blockchain technology is an immutable database that cannot be censored, tampered with, or erased, which can increase credibility and reduce interaction costs. Therefore, blockchain technology is expected to become the next generation Internet infrastructure, with broad application prospects in many industries, such as medicine, energy, and finance. The open network and publicly stored data in a blockchain system, however, raise serious risks of data theft and user privacy leakage, which has become a core issue that restricts the blockchain technology from becoming practical. This article uses kleptography technology to design a blockchain covert data transmission scheme to achieve high-concealment and high-performance data transmission under open network conditions. Experiments conducted in the Bitcoin network prove that the covert data transmission mechanism of this article has high compatibility and high performance, and can be practically applied to many popular blockchain systems.

DeepNFV: A Lightweight Framework for Intelligent Edge Network Functions Virtualization

January 4, 2023/in Publikasi /by Informatika

DeepNFV: A Lightweight Framework for Intelligent Edge Network Functions Virtualization

by Liangzhi Li, Kaoru Ota, Mianxiong Dong

Institute of Electrical and Electronics Engineers (IEEE)

Traditional Network Functions Virtualization (NFV) implementations are somehow too heavy and do not have enough functionality to conduct complex tasks. In this work, we propose a lightweight NFV framework named DeepNFV, which is based on the Docker container running on the network edge, and integrates state-of-the-art deep learning models with NFV containers to address some complicated problems, such as traffic classification, link analysis, and so on. We compare the DeepNFV framework with several existing works, and detail its structures and functions. The most significant advantage of DeepNFV is its lightweight design, resulting from the virtualization and low-cost nature of the container technology. Also, we design this framework to be compatible with edge devices, in order to decrease the computational overhead of the central servers. Another merit is its strong analysis ability brought by deep learning models, which make it suitable for many more scenarios than traditional NFV approaches. In addition, we also describe some typical application scenarios, regarding how the NFV container works and how to utilize its learning ability. Simulations demonstrate its high efficiency, as well as the outstanding recognition performance in a typical use case.

End-to-End Network Slicing in Radio Access Network, Transport Network and Core Network Domains

January 4, 2023/in Publikasi /by Informatika

End-to-End Network Slicing in Radio Access Network, Transport Network and Core Network Domains

by Xu Li, Rui Ni, Jun Chen, Yibo Lyu, Zhichao Rong, Rui Du

Institute of Electrical and Electronics Engineers (IEEE)

Network slicing (NS) has been well discussed in the transport network (TN) and core network (CN) domains. This paper extends it to the radio access network (RAN) domain, and the NS in RAN, TN and CN domains is defined as end-to-end (E2E) NS system. The advantages of using NS in the RAN domain with two-level resource allocation scheme are studied and shown by numerical simulations. Then the E2E NS system architecture and components are proposed and demonstrated with hardware and software. The demonstration shows the capability with very fine spectral granularity, and the slice creation, delete and adjustment schemes in sub-minute time, which could be used in the operator’s network.

Network Slicing-Based Customization of 5G Mobile Services

January 4, 2023/in Publikasi /by Informatika

Network Slicing-Based Customization of 5G Mobile Services

by Ibrahim Afolabi, Tarik Taleb, Pantelis A. Frangoudis, Miloud Bagaa, Adlen Ksentini

Institute of Electrical and Electronics Engineers (IEEE)

Through network slicing, different requirements of different applications and services can be met. These requirements can be in terms of latency, bandwidth, mobility support, defining service area, and security. Through fine and dynamic tuning of network slices, services can have their delivery platforms constantly customized according to their changing needs. In this article, we present our implementation of an E2E network slice orchestration platform, evaluate its performance in terms of dynamic deployment of network slices in an E2E fashion, and discuss how its functionality can be enhanced to better customize the network slices according to the needs of their respective services.

Key Points

A comprehensive survey on graph neural networks

Program Studi Informatika

Informasi Tentang