Research | Current

• Adversarial Attacks
• Adversarial Detection
• Model Robustness
• Privacy-Aware Learning
• Causality Inference
• Anomaly Detection

Adversarial Attacks

Deep neural networks (DNN) have made significant progresses in a wide domain of machine learning, including image classification, object detection, automatic speech recognition, content-based image retrieval (CBIR), steganography, and more. DNN provides an end-to-end learning approach that automates feature extraction with higher and more abstract level representation from the raw input. Recently, deep learning is found to be vulnerable under adversarial attacks. In particular, malicious input can be generated by imposing adversarial perturbations on the original input. Such adversarial examples are designed to induce wrongful decisions of the system model and are often imperceptible to human eyes. Currently, we are working on a number of query-efficient black-box attacks on different DNN-based application systems. Figure 1 illustrates an example of adversarial attack on CBIR systems, where the top-K retrieval results are subverted by an adversarial example of the query image.

Figure 1 An example of adversarial attack on CBIR systems.

Adversarial Detection

Adversarial detection attempts to distinguish adversarial examples from the normal inputs. The goal is to maximize the chance of allowing only the legitimate input to the intended model. Adversarial detection methods may be classified into model-dependent and model-agnostic approaches depending on interacting with the intended model or not. The dependent schemes often leverage the underlying model properties or internal states to detect the adversarial class, e.g., by adding detection layers/subnetworks or changing the loss/activation function. The model-agnostic detectors are mainly built based on analyzing the input and/or output feature characteristics without requiring access to the intended model. Currently, we are working on model-agnostic methods for adversarial detection at different layers of DNN. For example, we apply random perturbations to the model input for multiple times and use the statistical patterns of relative changes in the model output for adversarial detection. Figure 2 shows the detection accuracy of this method which is particularly effective for detecting small adversarial perturbations.

Figure 2 Adversarial detection accuracy w.r.t. adversarial perturbation strength.

Model Robustness

Adversarial defense aims at improving the model robustness against adversarial attacks. Many defense techniques have been proposed in recent years. Popular defense paradigms include randomization techniques and training with regularizations. However, many of them are later shown defeated by stronger iterative attacks or adaptive adversaries. So far, adversarial training has been considered as a standard method for defending against adversarial examples by data augmentation. However, adversarial training requires to generate large volumes of adversarial examples during the training phase, which introduces a high computational complexity. In general, there is a trade-off between classification accuracy and adversarial robustness in many defense methods. We are currently working on more effective methods of adversarial training by reducing its cost and improving its generalizability. Figure 3 demonstrates our method of improving model robustness by promoting diversified simultaneous training of deep ensembles in comparison with STOA.

Figure 3 Improving model robustness by promoting diversified learning of deep features in deep ensembles.

Privacy-Aware Learning

Privacy-aware machine learning includes two aspects: 1) privacy leakage, and 2) privacy protection. In privacy leakage, an attacker intends to derive sensitive information from machine learning models. The privacy leakage problems include membership inference, data property inference, sample reconstruction, and model extraction. In privacy protection, one studies the countermeasures of how to prevent machine learning models from leaking privacy, e.g., differential privacy, model compression, selective sharing and more. We are currently studying privacy and security issues in deep learning, including training data privacy and model privacy. Examples are various inference attacks in collaborative learning and inference-based similarity search for privacy-preserving biometric identification. Figure 4 demonstrates three different categories of research problems in privacy and machine learning (ML): (a) Model privacy and data privacy; (b) ML-enhanced privacy protection; (c) ML-based privacy attack.

Figure 4 Research problems in privacy and ML.

Causality Inference

Causality is a generic relationship between an effect and the cause that gives rise to it. It is hard to define, and even human being often only know intuitively about causes and effects. When it comes to learning causality with data, researchers need to be aware of the differences between statistical associations and causations. As shown in Figure 5, the statistical dependence reflected from deep features is not equal to the relationship of causation. Whereas the causation is likely to infer statistical dependence. It contains spurious and causal relationship in the statistical associations. Causality inference helps interpretability of machine learning and has been used in a number of fields including supervised learning, semi-supervised learning, domain adaptation, reinforcement learning, disentanglement learning and so on.

Figure 5 The relationship between causation and statistical dependence.

Anomaly Detection

Anomaly detection is an important topic in data mining to identify data points, events, and/or observations that deviate from the data’s normal behavior. Anomalous data can indicate critical incidents, such as a technical glitch, or potential opportunities, e.g., a change in consumer behavior. We are currently developing one-class classifiers for anomaly detection. We are also interested in studying the relationship between anomaly detection and adversarial examples. Figure 6 illustrates the subtle difference of these two types.

Figure 6 Difference of OOD examples and adversarial examples.