Within the context of the Belt and Road Initiative (BRI), which aims to connect China and Europe, the role and exercise of public diplomacy in China-Europe relations is becoming increasingly significant, in particular China’s public diplomacy in Europe. China has had some success in exercising its public diplomacy in Europe, but the means of improving its relationship with Europe through public diplomacy strategy within the context of BRI have been discussed less. China has been investing heavily in its charm offensive, and it does have an increasingly positive influence on developing countries, especially in Africa and Latin America. However, its image in Europe is still predominantly negative. Due to the differences in cultural norms and values between China and Europe, it is not enough for China to use its traditional culture and history to create soft power; instead, it needs to unleash the talents of its civil society and make its attractiveness conform to domestic realities. This article aims to tackle problems with the exercise of China’s public diplomacy in Europe and provide practical suggestions on how to enhance it in the future, particularly through the use of non-governmental organisations (NGOs). To this end, working within the theory of constructivism, it proposes that China and Europe must go beyond the emotional entanglements of history and culture, cultivate innovative thinking and initiate a new paradigm in China-Europe relations.