ChatGPT作为旅游研究问卷调查工具：优势与局限

Abstract

Abstract: This research discusses the advantages and limitations of using ChatGPT as a questionnaire tool in tourism studies. By replicating the experimental design of a published article, this research compares the questionnaire data and analytical results obtained through traditional methods with those generated by ChatGPT. The study found that ChatGPT, when used appropriately, can assist researchers in conducting experimental research and questionnaire surveys. Compared with traditional small-sample surveys and experiments, ChatGPT generates responses that are more independently distributed, with better representativeness, and are well-suited for sentiment analysis. However, the responses generated by ChatGPT tend to be overly concentrated, lack randomness, and provide insufficient details. Researchers can use ChatGPT as a useful assistant in pilot studies or exploratory research in developing scales, revising experimental and sampling methods. This can help ensure sample quality and research validity, while also significantly reducing time and improving research efficiency. Nevertheless, researchers should critically assess when using AI tools and critically evaluate the quality and applicability of the AI-generated data in light of their specific research questions.

Key words: ChatGPT-3.5, experimental approach, questionnaire survey, reliability and validity, sentiment analysis, pilot study

CLC Number:

ZUO Bing, WU Jiao, YANG Yi. ChatGPT as a Questionnaire Survey Tool in Tourism Research：Strengths and Limitations[J].TOURISM SCIENCE, 2025, 39(10): 61-77.

References

[1] 博登,2001.人工智能哲学[M].刘西瑞,王汉琦,译.上海:上海译文出版社:94-95.
[2] 陈昌凤,张梦,2023.由数据决定?AIGC的价值观和伦理问题[J].新闻与写作(4):15-23.
[3] 陈静远,胡丽雅,吴飞,2023.ChatGPT/生成式人工智能促进以知识点为核心的教学模式变革研究[J].华东师范大学学报(教育科学版)(7):177-186.
[4] 陈强,2020.机器学习与R语言[M].北京:高等教育出版社.
[5] 丁文杰,2023.通用人工智能视野下著作权法的逻辑回归——从“工具论”到“贡献论”[J].东方法学(5):94-105.
[6] 董春雨,2023.从机器认识的不透明性看人工智能的本质及其限度[J].中国社会科学(5):148-166.
[7] 恩格斯,1883.《马克思恩格斯全集》(中文第二版)第十九卷[M/OL].中央编译局编译.北京:人民出版社.[2024-03-12].https://www.marxists.org/chinese/engels/mia-chinese-engels-18830317.htm.
[8] 郭小东,2023.生成式人工智能的风险及其包容性法律治理[J].北京理工大学学报(社会科学版)(6):93-105;117.
[9] 李骁天,胡良平,尚熙明,等,2023.大型语言模型对体育科学研究的重塑:应用与启示——以ChatGPT为例[J].西安体育学院学报(4):405-415.
[10] 秦明利,黄舒婷,2023.人工智能自然语言能力的物质基础与功能实证[J].长白学刊(6):42-51.
[11] 秦涛,杜尚恒,常元元,等,2024.ChatGPT的工作原理、关键技术及未来发展趋势[J].西安交通大学学报(1):1-12.
[12] 王一博,郭鑫,刘智锋,等,2023.AI生成与学者撰写中文论文摘要的检测与差异性比较研究[J].情报杂志(9):127-134.
[13] 王英明,汪怀君,2023.象脑与共情:ChatGPT的伦理风险发微与纠偏[J].理论导刊(7):108-113.
[14] 魏航,郑迎迎,陈静,2023.ChatGPT对商科的挑战与机遇[J].外国经济与管理(9):69-78.
[15] 香港科技大学,2023.生成式AI开创AI新世代[EB/OL].[2023-11-21].https://hkust.edu.hk/zh-hant/news/research-and-innovation/generative-ai-generation-ai.
[16] 曾建华,2023.人工智能与人文学术范式革命——来自ChatGPT的挑战与启示[J].北京师范大学学报(社会科学版)(4):78-88.
[17] 湛东升,张文忠,余建辉,等,2016.问卷调查方法在中国人文地理学研究的应用[J].地理学报(6):899-913.
[18] 赵晓伟,祝智庭,沈书生,2023.教育提示语工程:构建数智时代的认识论新话语[J].中国远程教育(11):22-31.
[19] 政光景,吕鹏,2023.生成式人工智能与哲学社会科学新范式的涌现[J].江海学刊(4):132-142.
[20] 中国新闻网,2023.《国内C刊声明:隐瞒ChatGPT使用情况将被退稿或撤稿》[EB/OL].中国新闻网微博.[2023-11-21].https://weibo.com/1784473157/MsELO2vx9.
[21] 朱光辉,王喜文,2023.ChatGPT的运行模式、关键技术及未来图景[J].新疆师范大学学报(哲学社会科学版)(4):113-122.
[22] ALI F,YASAR B,ALI L,et al.,2023. Antecedents and consequences of travelers’ trust towards personalized travel recommendations offered by ChatGPT[J/OL].International journal of hospitality management,114:103588. DOI:10.1016/j.ijhm.2023.103588.
[23] BENICHOU L,CHATGPT,2023. The role of using ChatGPT AI in writing medical scientific articles[J/OL].Journal of stomatology,oral and maxillofacial surgery,124(5):101456.DOI:10.1016/j.jormas.2023.101456.
[24] BROWN T B,MANN B,RYDER N,et al.,2020.Language models are few-shot learners[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems.Red Hook,NY,USA:Curran Associates Inc.:1877-1901.
[25] BYUN J,JANG S S,2015.Effective destination advertising:matching effect between advertising language and destination type[J].Tourism management,50:31-40.
[26] CARVALHO I,IVANOV S,2023.ChatGPT for tourism:applications,benefits and risks[J].Tourism review,79(2):1-14.
[27] CURTIS N,CHATGPT,2023.To ChatGPT or not to ChatGPT?The impact of artificial intelligence on academic publishing[J].The pediatric infectious disease journal,42(4):275.
[28] DEMIR M,DEMIR S S,2023.Is ChatGPT the right technology for service individualization and value co-creation?Evidence from the travel industry[J].Journal of travel & tourism marketing,40(5):383-398.
[29] DERGAA I,CHAMARI K,ZMIJEWSKI P,et al.,2023. From human writing to artificial intelligence generated text:examining the prospects and potential threats of ChatGPT in academic writing[J].Biology of sport,40(2):615-622.
[30] DOWLING M,LUCEY B,2023.ChatGPT for (finance) research:the Bananarama conjecture[J/OL].Finance research letters,53:103662.DOI:10.1016/j.frl.2023.103662.
[31] DWIVEDI Y K,PANDEY N,CURRIE W,et al.,2023.Leveraging ChatGPT and other generative artificial intelligence(AI)-based applications in the hospitality and tourism industry:practices,challenges and research agenda[J].International journal of contemporary hospitality management,36(1):1-12.
[32] FUSTÉ-FORNÉ F,OREA-GINER A,2023.Gastronomy in tourism management and marketing:an interview with ChatGPT[J].Robonomics:the journal of the automated economy,4:1-13.
[33] GHASEMI SIANI M,MOHAMMADI S,SOLTAN HOSSEINI M,et al.,2021.Comparing young adult responses to rational and emotional sports product advertisements:the moderating role of product type and gender[J].International journal of sports marketing and sponsorship,22(4) :798-815.
[34] GILARDI F,ALIZADEH M,KUBLI M,2023.ChatGPT outperforms crowd workers for text-annotation tasks[J].Proceedings of the national academy of sciences,120(30):1-3.
[35] GURSOY D,LI Y,SONG H,2023.ChatGPT and the hospitality and tourism industry:an overview of current trends and future research directions[J].Journal of hospitality marketing & management,32(5):579-592.
[36] ISKENDER A,2023.Holy or unholy?Interview with open AI’s ChatGPT[J].European journal of tourism research,34:3414.
[37] LI B,FANG G,YANG Y,et al.,2023.Evaluating ChatGPT’s information extraction capabilities:an assessment of performance,explainability,calibration,and faithfulness[J/OL].Arxiv preprint arxiv:2304.11633.DOI:10.48550/arXiv.2304.11633.
[38] LUND B D,WANG T,MANNURU N R,et al.,2023.ChatGPT and a new academic reality:artificial intelligence-written research papers and the ethics of the large language models in scholarly publishing[J].Journal of the association for information science and technology,74(5):570-581.
[39] OPENAI,2022.Introducing ChatGPT [EB/OL].[2023-11-21].https://openai.com/blog/chatgpt.
[40] OPENAI,2024.Video generation models as world simulators[EB/OL].[2024-3-11].https://openai.com/research/video-generation-models-as-world-simulators.
[41] SEGHIER M L,2023.ChatGPT:not all languages are equal[J].Nature,615(7951):216.
[42] SHIN H,KANG J,2023.Bridging the gap of bibliometric analysis:the evolution,current state,and future directions of tourism research using ChatGPT[J].Journal of hospitality and tourism management,57:40-47.
[43] STOKEL-WALKER C,2023.ChatGPT listed as author on research papers:many scientists disapprove[J].Nature,613(7945):620-621.
[44] WONG I A,LIAN Q L,SUN D,2023.Autonomous travel decision-making:an early glimpse into ChatGPT and generative AI[J].Journal of hospitality and tourism management,56:253-263.
[45] XU C,GUO D,DUAN N,et al.,2023.Baize:an open-source chat model with parameter-efficient tuning on self-chat data[J/OL].Arxiv preprint arxiv:2304.01196.DOI:10.48550/arXiv.2304.01196.
[46] YANG W,CHEN Q,HUANG X,et al.,2022.Image and text presentation forms in destination marketing:an eye-tracking analysis and a laboratory experiment[J/OL].Frontiers in psychology,13:1024991.DOI:10.3389/fpsyg.2022.1024991.

ChatGPT as a Questionnaire Survey Tool in Tourism Research：Strengths and Limitations

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 6

Metrics

Comments

Recommended 0

[1]	YU　Changyue, WANG　Xiaofeng. Still in Body，Recalling in Mind：Content Characteristics，Emotional Responses of Individual Travel Memories [J]. TOURISM SCIENCE, 2025, 39(7): 112-129.
[2]	LAI Xiaofan, XIE Jiaqian, WANG Xinrui. Quality Factors of Rural Tourist Attraction and Their Impacts on Tourist Satisfaction：Based on Kano Model with Explainable Neural Network [J]. TOURISM SCIENCE, 2025, 39(4): 120-139.
[3]	XIAO Lu, LI Qiaoxing, CHEN Yimeng, SHEN Jiasheng, ZHANG Qian, YANG Yong. Fine-Grained Sentiment Analysis of Tourists' Reviews Big Data： Based on Aspect Sentiment Triplet Extraction [J]. TOURISM SCIENCE, 2024, 38(8): 60-87.
[4]	LIU Yi, XIAO Wenjie, GUAN Xi, GAO Xuan, ZHENG Ziyu. Exploring Language Positivity Bias in Chinese Online Tourism Reviews [J]. TOURISM SCIENCE, 2024, 38(11): 48-66.
[5]	SHI Da, WANG Zhimin. On Green Hotel User Experience:A Deep Learning Study Based on Online Reviews [J]. TOURISM SCIENCE, 2019, 33(6): 62-76.
[6]	. A Comparative Study on the Data of a Questionnaire and Web Texts in Tourism Image Researches: A Case Study of Xi’an’s Tourism Image Perception [J]. TOURISM SCIENCE, 2014, 28(6): 73-81.