Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping

Tianyi Xu; Ozge Ilkim Ozbek; Shannon Marks; Sri Korrapati; Benyamin Ahmadnia

doi:doi:10.11648/j.ajai.20200402.11

| Peer-Reviewed

Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping

Tianyi Xu, Ozge Ilkim Ozbek, Shannon Marks, Sri Korrapati, Benyamin Ahmadnia

Published in American Journal of Artificial Intelligence (Volume 4, Issue 2)

Received: 28 May 2020 Accepted: 18 June 2020 Published: 23 July 2020

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

The quality of data-driven Machine Translation (MT) strongly depends on the quantity as well as the quality of the training dataset. However, collecting a large set of training parallel texts is not easy in practice. Although various approaches have already been proposed to overcome this issue, the lack of large parallel corpora still poses a major practical problem for many language pairs. Since monolingual data plays an important role in boosting fluency for Neural MT (NMT) models, this paper investigates and compares the performance of two learning-based translation approaches for Spanish-Turkish translation as a low-resource setting in case we only have access to large sets of monolingual data in each language; 1) Unsupervised Learning approach, and 2) Round-Tripping approach. Either approach completely removes the need for bilingual data or enables us to train the NMT system relying on monolingual data only. We utilize an Attention-based NMT (Attentional NMT) model, which leverages a careful initialization of the parameters, the denoising effect of language models, and the automatic generation of bilingual data. Our experimental results demonstrate that the Unsupervised Learning approach outperforms the Round-Tripping approach in Spanish-Turkish translation and vice versa. These results confirm that the Unsupervised Learning approach is still a reliable learning-based translation technique for Spanish-Turkish low-resource NMT.

Published in	American Journal of Artificial Intelligence (Volume 4, Issue 2) This article belongs to the Special Issue Machine Translation for Low-Resource Languages
DOI	10.11648/j.ajai.20200402.11
Page(s)	42-49
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2020. Published by Science Publishing Group

Keywords

Computational Linguistics, Natural Language Processing, Neural Machine Translation, Low-Resource Languages, Unsupervised Learning, Round-Tripping

References

[1]	Lample G., Conneau A., Denoyer L., Ranzato M., Unsupervised machine translation using monolingual corpora only, Proceedings of the International Conference on Learning Representations, 2018.
[2]	Bahdanau D., Cho K., Bengio Y., Neural machine translation by jointly learning to align and translate, Proceedings of the International Conference on Learning Representations, 2015.
[3]	Ahmadnia B., Haffari G., Serrano J., Round-trip training approach for bilingually low-resource statistical machine translation systems, International Journal of Artificial Intelligence, 2019, 17 (1): 167-185.
[4]	Ahmadnia B., Dorr B. J., Augmenting Neural Machine Translation through Round-Trip Training Approach, Open Computer Science, 2019, 9 (1): 268-278.
[5]	Ahmadnia B., Haffari G., Serrano J., Statistical Machine Translation for Bilingually Low-Resource Scenarios: A Round-Tripping Approach, Proceedings of the 3rd IEEE International Conference on Machine Learning and Natural Language Processing, 2018, 261-265.
[6]	He D., Xia Y., Qin T., Wang L., Yu N., Liu T., Ma W., Dual learning for machine translation, Proceedings of the 30th Conference on Neural Information Processing Systems, 2016.
[7]	Wu H., Wang H., Pivot language approach for phrase-based statistical machine translation, Proceedings of ACL: the 45th Annual Meeting of the Association of Computational Linguistics, 2007, 856-863.
[8]	Ahmadnia B., Serrano J., Direct translation vs. pivot language translation for Persian-Spanish low-resourced statistical machine translation system, Proceedings of the 18th International Conference on Artificial Intelligence and Computer Science, 2016.
[9]	Ahmadnia B., Serrano J., Haffari G., Persian-Spanish low-resource statistical machine translation through English as pivot language, Proceedings of Recent Advances in Natural Language Processing, 2017, 24-30.
[10]	Ahmadnia B., Serrano J., Employing pivot language technique through statistical and neural machine translation frameworks: The case of under-resourced Persian-Spanish language pair, International Journal on Natural Language Computing, 2017, 6 (5): 37-47.
[11]	Ahmadnia B., Serrano, Haffari G., Balouchzahi NM., Direct-bridge combination scenario for Persian-Spanish low-resource statistical machine translation, Proceedings of Artificial Intelligence and Natural Language, 2018, 67-78.
[12]	Firat O., Sankaran B., Al-Onaizan Y., Yarman Vural F. T., Cho K., Effective approaches to attention-based neural machine translation, Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016, 268–277.
[13]	Chen, Y., Liu Y., Cheng Y., Li V., A teacher-student framework for zero- resource neural machine translation, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017, 1925-1935.
[14]	Sennrich R., Haddow B., Birch A., Improving neural machine translation models with monolingual data, Proceedings of the 54th Annual Meeting of Association for Computational Linguistics, 2016.
[15]	Luong T., Sutskever I., Le Q. V., Vinyals O., Zaremba W., Addressing the rare word problem in neural machine translation, Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 2015, 11–19.
[16]	Koehn P., Knowles R., Six Challenges for Machine Translation, Proceedings of the First Workshop on Neural Machine Translation, 2017.
[17]	Sennrich R., Zhang B., Revisiting Low-Resource Neural Machine Translation: A Case Study, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, 211-221.
[18]	Lample G., Ott M., Conneau A., Denoyer L., Ranzato M., Phrase-based & neural unsupervised machine translation, Proceedings of EMNLP, 2018, 5039-5049.
[19]	Imankulova, A., Sato, T., Komachi, M., Improving Low-Resource Neural Machine Translation with Filtered Pseudo-parallel Corpus, Proceedings of the 4th Workshop on Asian Translation, 2017, 70-78.
[20]	Fadaee M., Bisazza A., Monz C., Data augmentation for low-resource neural machine translation, Association for Computational Linguistics, 2019.
[21]	Artetxe, M., Labaka, G., Agirre, E., & Cho, K. 2017. Unsupervised neural machine translation. arXiv preprint arXiv: 1710.11041.
[22]	Artetxe M., Labaka G., Agirre E., An effective approach to unsupervised machine translation, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, 194-203.
[23]	Hochreiter S., Schmidhuber J., Long short-term memory, Neural Computation, 1997, 9 (8), 1735–1780.
[24]	Luong T., Pham H., Manning C. D., Effective approaches to attention-based neural machine translation, Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015, 1412–1421.
[25]	Tiedemann J., Parallel data, tools and interfaces in OPUS, Proceedings of the 8th International Conference on Language Resources and Evaluation, 2012.
[26]	Koehn P., Europarl: A parallel corpus for statistical machine translation, Proceedings of the 10th Machine Translation Summit, 2005, 79-86.
[27]	Sennrich R., Haddow B., Birch A., Neural machine translation of rare words with subword units, Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016b, 1715-1725.
[28]	Mikolov T., Sutskever I., Chen K., Corrado G., Dean J., Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26, 2013, 3111-3119.
[29]	Mi H., Wang Z., Ittycheriah A., Supervised attentions for neural machine translation, Proceedings of the International Conference on Empirical Methods in Natural Language Processing, 2016, 2283-2288.
[30]	Cohn T., Huang C. D. V., Vymolova E., Yao K., Dyer C., Haffari G., Incorporating structural alignment biases into an attentional neural translation model, Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies, 2016, 876-885.
[31]	Sutskever I., Vinyals O., le Q. V., Sequence to sequence learn- ing with neural networks, Proceedings of Advances in Neural Information Processing Systems, 2014, 3104-3112.
[32]	Papineni K., Roukos S., Ward T., Zhu W. J., BLEU: A method for automatic evaluation of machine translation, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, 2001, 311-318.
[33]	Ahmadnia B., Kordjamshidi P., Haffari G., Neural machine translation advised by statistical machine translation: The case of Farsi-Spanish bilingually low-resource scenario, Proceedings of the 17th IEEE International Conference on Machine Learning and Applications, 2018, 1209-1213.

Cite This Article

Plain Text BibTeX RIS

APA Style

Tianyi Xu, Ozge Ilkim Ozbek, Shannon Marks, Sri Korrapati, Benyamin Ahmadnia. (2020). Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping. American Journal of Artificial Intelligence, 4(2), 42-49. https://doi.org/10.11648/j.ajai.20200402.11

Copy | Download

ACS Style

Tianyi Xu; Ozge Ilkim Ozbek; Shannon Marks; Sri Korrapati; Benyamin Ahmadnia. Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping. Am. J. Artif. Intell. 2020, 4(2), 42-49. doi: 10.11648/j.ajai.20200402.11

Copy | Download

AMA Style

Tianyi Xu, Ozge Ilkim Ozbek, Shannon Marks, Sri Korrapati, Benyamin Ahmadnia. Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping. Am J Artif Intell. 2020;4(2):42-49. doi: 10.11648/j.ajai.20200402.11

Copy | Download

@article{10.11648/j.ajai.20200402.11,
  author = {Tianyi Xu and Ozge Ilkim Ozbek and Shannon Marks and Sri Korrapati and Benyamin Ahmadnia},
  title = {Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping},
  journal = {American Journal of Artificial Intelligence},
  volume = {4},
  number = {2},
  pages = {42-49},
  doi = {10.11648/j.ajai.20200402.11},
  url = {https://doi.org/10.11648/j.ajai.20200402.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajai.20200402.11},
  abstract = {The quality of data-driven Machine Translation (MT) strongly depends on the quantity as well as the quality of the training dataset. However, collecting a large set of training parallel texts is not easy in practice. Although various approaches have already been proposed to overcome this issue, the lack of large parallel corpora still poses a major practical problem for many language pairs. Since monolingual data plays an important role in boosting fluency for Neural MT (NMT) models, this paper investigates and compares the performance of two learning-based translation approaches for Spanish-Turkish translation as a low-resource setting in case we only have access to large sets of monolingual data in each language; 1) Unsupervised Learning approach, and 2) Round-Tripping approach. Either approach completely removes the need for bilingual data or enables us to train the NMT system relying on monolingual data only. We utilize an Attention-based NMT (Attentional NMT) model, which leverages a careful initialization of the parameters, the denoising effect of language models, and the automatic generation of bilingual data. Our experimental results demonstrate that the Unsupervised Learning approach outperforms the Round-Tripping approach in Spanish-Turkish translation and vice versa. These results confirm that the Unsupervised Learning approach is still a reliable learning-based translation technique for Spanish-Turkish low-resource NMT.},
 year = {2020}
}

Copy | Download

TY  - JOUR
T1  - Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping
AU  - Tianyi Xu
AU  - Ozge Ilkim Ozbek
AU  - Shannon Marks
AU  - Sri Korrapati
AU  - Benyamin Ahmadnia
Y1  - 2020/07/23
PY  - 2020
N1  - https://doi.org/10.11648/j.ajai.20200402.11
DO  - 10.11648/j.ajai.20200402.11
T2  - American Journal of Artificial Intelligence
JF  - American Journal of Artificial Intelligence
JO  - American Journal of Artificial Intelligence
SP  - 42
EP  - 49
PB  - Science Publishing Group
SN  - 2639-9733
UR  - https://doi.org/10.11648/j.ajai.20200402.11
AB  - The quality of data-driven Machine Translation (MT) strongly depends on the quantity as well as the quality of the training dataset. However, collecting a large set of training parallel texts is not easy in practice. Although various approaches have already been proposed to overcome this issue, the lack of large parallel corpora still poses a major practical problem for many language pairs. Since monolingual data plays an important role in boosting fluency for Neural MT (NMT) models, this paper investigates and compares the performance of two learning-based translation approaches for Spanish-Turkish translation as a low-resource setting in case we only have access to large sets of monolingual data in each language; 1) Unsupervised Learning approach, and 2) Round-Tripping approach. Either approach completely removes the need for bilingual data or enables us to train the NMT system relying on monolingual data only. We utilize an Attention-based NMT (Attentional NMT) model, which leverages a careful initialization of the parameters, the denoising effect of language models, and the automatic generation of bilingual data. Our experimental results demonstrate that the Unsupervised Learning approach outperforms the Round-Tripping approach in Spanish-Turkish translation and vice versa. These results confirm that the Unsupervised Learning approach is still a reliable learning-based translation technique for Spanish-Turkish low-resource NMT.
VL  - 4
IS  - 2
ER  -

Copy | Download

Author Information

Tianyi Xu

Department of Computer Science, Tulane University of Louisiana, New Orleans, United States
Ozge Ilkim Ozbek

Department of Linguistics, Tulane University of Louisiana, New Orleans, United States
Shannon Marks

Department of Linguistics, Tulane University of Louisiana, New Orleans, United States
Sri Korrapati

Department of Linguistics, Tulane University of Louisiana, New Orleans, United States
Benyamin Ahmadnia

Department of Computer Science, Tulane University of Louisiana, New Orleans, United States

Download PDF

Sections

Plain Text BibTeX RIS

APA Style

Tianyi Xu, Ozge Ilkim Ozbek, Shannon Marks, Sri Korrapati, Benyamin Ahmadnia. (2020). Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping. American Journal of Artificial Intelligence, 4(2), 42-49. https://doi.org/10.11648/j.ajai.20200402.11

Copy | Download

ACS Style

Tianyi Xu; Ozge Ilkim Ozbek; Shannon Marks; Sri Korrapati; Benyamin Ahmadnia. Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping. Am. J. Artif. Intell. 2020, 4(2), 42-49. doi: 10.11648/j.ajai.20200402.11

Copy | Download

AMA Style

Tianyi Xu, Ozge Ilkim Ozbek, Shannon Marks, Sri Korrapati, Benyamin Ahmadnia. Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping. Am J Artif Intell. 2020;4(2):42-49. doi: 10.11648/j.ajai.20200402.11

Copy | Download

@article{10.11648/j.ajai.20200402.11,
  author = {Tianyi Xu and Ozge Ilkim Ozbek and Shannon Marks and Sri Korrapati and Benyamin Ahmadnia},
  title = {Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping},
  journal = {American Journal of Artificial Intelligence},
  volume = {4},
  number = {2},
  pages = {42-49},
  doi = {10.11648/j.ajai.20200402.11},
  url = {https://doi.org/10.11648/j.ajai.20200402.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajai.20200402.11},
  abstract = {The quality of data-driven Machine Translation (MT) strongly depends on the quantity as well as the quality of the training dataset. However, collecting a large set of training parallel texts is not easy in practice. Although various approaches have already been proposed to overcome this issue, the lack of large parallel corpora still poses a major practical problem for many language pairs. Since monolingual data plays an important role in boosting fluency for Neural MT (NMT) models, this paper investigates and compares the performance of two learning-based translation approaches for Spanish-Turkish translation as a low-resource setting in case we only have access to large sets of monolingual data in each language; 1) Unsupervised Learning approach, and 2) Round-Tripping approach. Either approach completely removes the need for bilingual data or enables us to train the NMT system relying on monolingual data only. We utilize an Attention-based NMT (Attentional NMT) model, which leverages a careful initialization of the parameters, the denoising effect of language models, and the automatic generation of bilingual data. Our experimental results demonstrate that the Unsupervised Learning approach outperforms the Round-Tripping approach in Spanish-Turkish translation and vice versa. These results confirm that the Unsupervised Learning approach is still a reliable learning-based translation technique for Spanish-Turkish low-resource NMT.},
 year = {2020}
}

Copy | Download

TY  - JOUR
T1  - Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping
AU  - Tianyi Xu
AU  - Ozge Ilkim Ozbek
AU  - Shannon Marks
AU  - Sri Korrapati
AU  - Benyamin Ahmadnia
Y1  - 2020/07/23
PY  - 2020
N1  - https://doi.org/10.11648/j.ajai.20200402.11
DO  - 10.11648/j.ajai.20200402.11
T2  - American Journal of Artificial Intelligence
JF  - American Journal of Artificial Intelligence
JO  - American Journal of Artificial Intelligence
SP  - 42
EP  - 49
PB  - Science Publishing Group
SN  - 2639-9733
UR  - https://doi.org/10.11648/j.ajai.20200402.11
AB  - The quality of data-driven Machine Translation (MT) strongly depends on the quantity as well as the quality of the training dataset. However, collecting a large set of training parallel texts is not easy in practice. Although various approaches have already been proposed to overcome this issue, the lack of large parallel corpora still poses a major practical problem for many language pairs. Since monolingual data plays an important role in boosting fluency for Neural MT (NMT) models, this paper investigates and compares the performance of two learning-based translation approaches for Spanish-Turkish translation as a low-resource setting in case we only have access to large sets of monolingual data in each language; 1) Unsupervised Learning approach, and 2) Round-Tripping approach. Either approach completely removes the need for bilingual data or enables us to train the NMT system relying on monolingual data only. We utilize an Attention-based NMT (Attentional NMT) model, which leverages a careful initialization of the parameters, the denoising effect of language models, and the automatic generation of bilingual data. Our experimental results demonstrate that the Unsupervised Learning approach outperforms the Round-Tripping approach in Spanish-Turkish translation and vice versa. These results confirm that the Unsupervised Learning approach is still a reliable learning-based translation technique for Spanish-Turkish low-resource NMT.
VL  - 4
IS  - 2
ER  -

Copy | Download