Taiwan is charting a distinct path in artificial intelligence, prioritizing language models that mirror its unique cultural identity and democratic values, as a response to China’s advancements, particularly with DeepSeek-R1. This initiative aims to establish a counterbalance to AI systems influenced by authoritarian regimes.
The DeepSeek Challenge
The introduction of DeepSeek-R1 in January generated considerable discussion within the technology sector. While earlier Chinese language models, such as Baidu’s Ernie and ByteDance’s Doubao, demonstrated potential in Chinese language applications, mathematics, and coding, they were limited by weaker English proficiency and accessibility constraints. DeepSeek-R1, however, represented a notable achievement as the first Chinese LLM to gain international recognition.
A particularly noteworthy aspect of DeepSeek-R1 was its reportedly low development cost. Unlike OpenAI’s GPT-4o, which purportedly cost over US$100 million to train, DeepSeek researchers stated their chatbot was developed for just US$5.6 million. Further reinforcing the narrative of efficiency, DeepSeek engineers trained the R1 model using mid-range CPUs like the Nvidia H800, instead of the high-end chips used in models like GPT-4o or Anthropic’s Claude. Despite U.S. restrictions on exporting high-performance chips to China, DeepSeek-R1 reportedly outperformed other leading bots by utilizing only 2,048 processors across 256 servers.
This impressive efficiency and reduced development cost were largely attributed to advanced programming techniques, including PTX, an assembly-like language that enables developers to fine-tune performance and optimize hardware utilization.
Shortly after its release, the DeepSeek-R1 application climbed to the top of the U.S. Apple App Store’s free download rankings, surpassing ChatGPT, TikTok, and Meta’s social media platforms. The Nasdaq experienced a decline, and Nvidia’s shares fell following the debut of DeepSeek-R1.
Questioning DeepSeek’s Claims
Despite the initial excitement, numerous observers have questioned the accuracy of DeepSeek’s claims regarding its LLM. Analysts have suggested that the reported figures likely only account for computational costs, while excluding or understating infrastructure, hardware, and personnel expenses.
Wesley Kuo, founder and CEO of Ubitus, a Taipei-based generative AI and cloud gaming service provider, echoed these concerns, stating that the actual cost is probably much higher than what is being reported. Ubitus, with backing from Nvidia, supported Project TAME, a localized LLM using traditional Chinese characters. They provided H100 CPUs and gaming data. Ubitus also partnered with Foxlink and Shinfox Energy to establish Ubilink.AI, building Taiwan’s largest green energy-powered AI supercomputing service center in collaboration with Asus.
Kuo emphasized Ubitus’ involvement in developing LLM applications and models for governments, including the Japanese government, across sectors like gaming, tourism, and retail, highlighting AI’s potential to alleviate labor shortages and address aging populations. The application of AI spans diverse sectors and its impact is projected to grow exponentially in the coming years.
Data Integrity Concerns
Kuo aligns with OpenAI and Microsoft in suggesting that DeepSeek may have acquired data through model distillation. This process involves training smaller language models to mimic the outputs of larger models. OpenAI and Microsoft allege that DeepSeek used OpenAI’s application programming interface to facilitate its development.
Kuo asserts that DeepSeek obtained data from OpenAI and that there are misunderstandings surrounding the company’s claims about efficiency. He points out that DeepSeek-R1, with its 670 billion parameters, is considerably larger than Meta AI’s Llama 3.1 405B. The parameters are internal numerical values a model learns during training to make predictions. Kuo also suggests that DeepSeek’s models may have been distilled from Llama 3.1. Distillation is a common technique, but the core data origins remain a crucial point of inquiry when assessing the integrity of a model.
Beyond these rebuttals, concerns have also arisen regarding the capabilities of DeepSeek-R1. Experts suggest that, like its predecessors, R1 excels in specialized, task-specific functions but lags behind versions of GPT-4o in general-purpose performance. The trade-offs between specialization and general applicability are a constant consideration in AI development.
A major limitation of DeepSeek’s models is the restriction on free access to information. Users discovered that inquiries about sensitive political topics were met with evasive responses. On topics such as the status of Xinjiang’s Uyghur minority and Taiwan, DeepSeek’s responses reflect the official Chinese Communist Party positions. Research suggests that a significant portion of DeepSeek’s outputs are censored to suppress information related to democracy, human rights, and China’s contested sovereignty claims. The inherent biases and censorship within the model present a substantial challenge to its objective utility, especially when addressing politically sensitive topics.
Taiwan’s Alternative: TAIDE and Beyond
In response, Taiwan-developed LLMs, such as TAME, have emerged as alternatives to DeepSeek within the Sinosphere. The Trustworthy AI Dialogue Engine (TAIDE), launched in June 2023 by the National Institute of Applied Research, aims to develop a model aligned with Taiwan’s social, cultural, and linguistic norms.
While work on TAIDE appears to have stalled, it served as an important benchmark for Project TAME. TAME, developed by the Machine Intelligence and Understanding Laboratory (MiuLab) at National Taiwan University, with funding from various organizations, was trained on 500 billion tokens. It outperformed competitors, including GPT-4o, across 39 evaluations, achieving higher scores on university entrance, bar, and traditional Chinese medicine examinations. The focus on traditional Chinese medicine, for instance, reflects an effort to integrate unique aspects of Taiwanese culture and expertise.
One of TAME’s objectives is to promote local culture. Unlocking local language capabilities is a significant step. Kuo mentions the development of a Taiwanese voice LLM based on Whisper, which has achieved positive results in understanding oral Taiwanese. Efforts are underway to develop Hakka language recognition.
These efforts have been well-received by institutions in regions where these languages are prevalent. There are also efforts to train the model in indigenous language recognition, but limited data remains an obstacle. Training AI to learn a new language requires a significant amount of voice recordings paired with text. The scarcity of data for many indigenous languages poses a hurdle to integrating these languages into AI models.
Accessing historical data in government archives presents another opportunity. However, some data is protected by copyright. The emergence of artificial general intelligence offers the potential to aid in the revival of endangered and extinct languages. The potential of AI to assist in language preservation represents an exciting avenue for future development.
The Pursuit of AI Sovereignty
The intersection of language and culture underscores the importance of AI sovereignty as a means of reinforcing Taiwanese identity, communicating Taiwan’s narrative, and protecting its information environment. AI sovereignty is not merely about technological independence; it’s about ensuring that technology reflects and respects the values and culture of a particular nation.
Julian Chu, an industry consultant and director at the Market Intelligence & Consulting Institute (MIC), emphasizes the potential for bias in LLM models and training data. He notes that even when using traditional characters, LLM outputs can reflect the style of the People’s Republic of China and fail to capture Taiwan’s culture. The goal is for Taiwanese companies to use Taiwanese language or data to train LLMs and build AI sovereignty.
Chu mentions the Formosa Foundation Model (FFM-Llama2) as another promising Taiwan LLM. Released in September 2023 by Taiwan Web Service, it aimed to democratize AI. Foxconn also launched its LLM, FoxBrain, in March. However, some commentators remain skeptical of big corporations’ ventures into LLMs. Democratization in the AI space necessitates not only technological accessibility, but also the consideration of ethical implications and societal impact.
Lin Yen-ting, a member of the MiuLab team that developed TAME, emphasizes the need to address the gap in the information environment regarding Taiwan. He notes that DeepSeek-R1 and other Chinese LLMs present a distorted view of Taiwan. U.S.-developed models can also sometimes misrepresent Taiwan. Open-source models may not prioritize Taiwan, and the training data is dominated by China.
Therefore, it is important to selectively incorporate Taiwanese content and retrain it into the model. This proactive approach ensures that Taiwan’s unique cultural and linguistic landscape is accurately represented in the digital realm, fostering a sense of national identity and preserving its distinct heritage in the face of global AI development. This dedication to preserving the Taiwanese identity ensures that the island nation’s unique culture and values are not overshadowed by dominant narratives. Continuous effort and careful curation are essential to avoid the homogenizing effects of globalized AI.
The challenges inherent in this endeavor are substantial. Building a truly representative AI model requires a significant investment of resources, including access to vast datasets of localized content and expertise in natural language processing. Furthermore, the ongoing need to counter disinformation and biased information necessitates a continuous process of refinement and adaptation. The fight against misinformation is an ongoing task that demands vigilance and adaptation as new methods of dissemination emerge.
Despite these challenges, Taiwan’s commitment to AI sovereignty remains steadfast. The development of TAME and other localized LLMs represents a crucial step toward ensuring that the future of artificial intelligence reflects the island’s unique cultural identity, democratic values, and unwavering commitment to preserving its distinct place in the world. By prioritizing AI sovereignty, Taiwan is not only safeguarding its cultural heritage but also positioning itself as a key player in the global AI landscape, demonstrating that technological advancement can be aligned with the preservation of cultural identity and democratic principles. The commitment to cultural preservation underscores the holistic nature of technological advancement.
Continuing the Journey
The journey towards complete AI sovereignty is ongoing. Further research, development, and collaboration are crucial to overcome the challenges and ensure the long-term success of these initiatives. By continuing to prioritize AI sovereignty, Taiwan can create a digital landscape that truly reflects its unique cultural identity and democratic values, setting an example for other nations striving to maintain their distinct place in an increasingly interconnected world. International collaboration and shared learning will be essential in shaping the future of responsible AI development.
The development of AI models which accurately reflect diverse cultures requires significant investment and expertise. The pursuit of AI sovereignty is not merely a technological endeavor but a social, cultural, and political project. Taiwan’s efforts demonstrate the potential for AI to be a tool for cultural preservation and national identity in an era of globalization. Maintaining this balance requires continuous vigilance and proactive engagement with the evolving AI landscape.