Differential privacy protection method for deep learning based on WGAN feedback

TAO Tao; BAI Jianshu; LIU Heng; HOU Shudong; ZHENG Xiao

doi:10.3969/j.issn.0253-2778.2020.08.004

JUSTC > 2020 > 50(8): 1064. > DOI: 10.3969/j.issn.0253-2778.2020.08.004

PDF (4753 KB)

Open Access JUSTC Original Paper

Differential privacy protection method for deep learning based on WGAN feedback

Cite this: JUSTC, 2020, 50(8): 1064-1071

https://doi.org/10.3969/j.issn.0253-2778.2020.08.004

Funds: Supported by the Key Research and Development Program Project of Anhui Province of China(201904d07020020), the Natural Science Foundation Project of Anhui Province of China(1908085MF212, 2008085MF190, 1808085QF210)， the Program for Synergy Innovation in the Anhui Higher Education Institutions of China(GXXT-2020-012).

More Information

Corresponding author:
TAO Tao(corresponding author), male, born in 1977, PhD/Associate Professor.
Received Date: June 04, 2020
Revised Date: August 17, 2020
Accepted Date: August 17, 2020
Published Date: August 30, 2020

Full text PDF

Abstract

Abstract

Aiming at the problem that attackers may steal sensitive information of the deep learning training dataset by some technological means such as the Generative Adversarial Network(GAN), combining the differential privacy theory, the differential privacy protection method was proposed for deep learning based on the Wasserstein generative adversarial network(WGAN) feedback parameter tuning. This privacy protection method is realized by optimization of the stochastic gradient descent, gradient clipping of setting gradient threshold, and noise adding to the optimization process of deep learning; WGAN was used to generate optimized results similar to the original data. The difference of the generated results and the original data were used for feedback parameter tuning. The experiment result shows that this method can effectively protect sensitive private information of the dataset and has preferable data usability.

Abstract

Aiming at the problem that attackers may steal sensitive information of the deep learning training dataset by some technological means such as the Generative Adversarial Network(GAN), combining the differential privacy theory, the differential privacy protection method was proposed for deep learning based on the Wasserstein generative adversarial network(WGAN) feedback parameter tuning. This privacy protection method is realized by optimization of the stochastic gradient descent, gradient clipping of setting gradient threshold, and noise adding to the optimization process of deep learning; WGAN was used to generate optimized results similar to the original data. The difference of the generated results and the original data were used for feedback parameter tuning. The experiment result shows that this method can effectively protect sensitive private information of the dataset and has preferable data usability.

FullText(HTML)

References (14)

References

[1]	HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1554.
[2]	SWEENEY L. K-anonymity: A model for protecting privacy[J]. International Journal of Uncertainty: Fuzziness and Knowledge-Based Systems, 2002, 10(05):557-570.
[3]	MACHANAVAJJHALA A, KIFER D, GEHRKE J. L-diversity: Privacy beyond k -anonymity[J]. Acm Transactions on Knowledge Discovery from Data, 2007, 1(1): 3.
[4]	LI N, LI T, VENKATASUBRAMANIAN S. t-Closeness: Privacy Beyond k-Anonymity and l-Diversity[C]//IEEE 23rd International Conference on Data Engineering. Piscataway: IEEE, 2007.
[5]	DWORK C. Differential privacy[J]. Lecture Notes in Computer Science, 2006, 26(2):1-12.
[6]	ABADI M, GOODFELLOW I, GOODFELLOW I, et al. Deep Learning with Differential Privacy[C]// ACM Sigsac Conference on Computer & Communications Security. 2016.
[7]	PAPERNOT N, ABADI M, ERLINGSSON L, et al. Semi-supervised knowledge transfer for deep learning from private training data[C/OL]. (2017-03-03)[2020-05-05]. https://arxiv.org/pdf/1610.05755v4.
[8]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]// International Conference on Neural Information Processing Systems. 2014.
[9]	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. Computer Science, 2015.
[10]	ULLOA A, BASILE A, WEHNER G J, et al. An unsupervised homogenization pipeline for clustering similar patients using electronic health record data [C/OL]. (2018-03-21)[2020-05-05]. https://arxiv.org/pdf/1801.00065.
[11]	XIE L, LIN K, WANG S, et al. Differentially private generative adversarial network [C/OL]. (2018-02-19)[2020-05-05]. https://arxiv.org/pdf/1802.06739.
[12]	HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep models under the GAN: Information leakage from collaborative deep learning[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York: Association for Computing Machinery, 2017: 603-618.
[13]	ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[C/OL]. (2017-12-06)[2020-05-05]. https://arxiv.org/pdf/1701.07875.
[14]	LECUN Y, CORTES C, BURGES C J. The MNIST database of handwritten digits[DB/OL].[2020-05-05]. http://yann.lecun.com/exdb/mnist/?o=3510.)

Cited By

Track Citations

Get Citation

{{if article.articleBusiness.pdfLink && article.articleBusiness.pdfLink != ''}} {{else}} {{/if}}PDF

XML

References

[1]	HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1554.
[2]	SWEENEY L. K-anonymity: A model for protecting privacy[J]. International Journal of Uncertainty: Fuzziness and Knowledge-Based Systems, 2002, 10(05):557-570.
[3]	MACHANAVAJJHALA A, KIFER D, GEHRKE J. L-diversity: Privacy beyond k -anonymity[J]. Acm Transactions on Knowledge Discovery from Data, 2007, 1(1): 3.
[4]	LI N, LI T, VENKATASUBRAMANIAN S. t-Closeness: Privacy Beyond k-Anonymity and l-Diversity[C]//IEEE 23rd International Conference on Data Engineering. Piscataway: IEEE, 2007.
[5]	DWORK C. Differential privacy[J]. Lecture Notes in Computer Science, 2006, 26(2):1-12.
[6]	ABADI M, GOODFELLOW I, GOODFELLOW I, et al. Deep Learning with Differential Privacy[C]// ACM Sigsac Conference on Computer & Communications Security. 2016.
[7]	PAPERNOT N, ABADI M, ERLINGSSON L, et al. Semi-supervised knowledge transfer for deep learning from private training data[C/OL]. (2017-03-03)[2020-05-05]. https://arxiv.org/pdf/1610.05755v4.
[8]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]// International Conference on Neural Information Processing Systems. 2014.
[9]	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. Computer Science, 2015.
[10]	ULLOA A, BASILE A, WEHNER G J, et al. An unsupervised homogenization pipeline for clustering similar patients using electronic health record data [C/OL]. (2018-03-21)[2020-05-05]. https://arxiv.org/pdf/1801.00065.
[11]	XIE L, LIN K, WANG S, et al. Differentially private generative adversarial network [C/OL]. (2018-02-19)[2020-05-05]. https://arxiv.org/pdf/1802.06739.
[12]	HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep models under the GAN: Information leakage from collaborative deep learning[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York: Association for Computing Machinery, 2017: 603-618.
[13]	ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[C/OL]. (2017-12-06)[2020-05-05]. https://arxiv.org/pdf/1701.07875.
[14]	LECUN Y, CORTES C, BURGES C J. The MNIST database of handwritten digits[DB/OL].[2020-05-05]. http://yann.lecun.com/exdb/mnist/?o=3510.)

TrendMD

Volume 50 Issue 8 PP. 1064-1071

Cover

Keywords

Article Metrics

Article views (150) PDF downloads (410)

Differential privacy protection method for deep learning based on WGAN feedback

Abstract

Abstract

References

Catalog

References

Related Articles

TrendMD

Article Metrics

Authors

Browse

Contact Us

About

Differential privacy protection method for deep learning based on WGAN feedback

Share

Tools

Abstract

Abstract

References

Catalog

References

Related Articles

TrendMD

Article Metrics

Authors

Browse

Contact Us

About

Export File

Citation

Format

Content