Blending Evidence and Wisdom–An Integrated Approach to Innovation Policy

John H. Howard, 29 April 2025

Evidence-based policy (EBP) and the theoretical foundations of neoclassical economics have become dominant frameworks in research and innovation policymaking. These approaches have significantly strengthened policymaking by ensuring decisions are guided by empirical data rather than ideology or political expediency—a development this paper strongly endorses.

However, while acknowledging the substantial benefits of EBP, this Insight argues that its exclusive reliance on measurable evidence (empiricism) can inadvertently undervalue the contributions of critical thinkers and visionaries whose insights often resist quantification. This creates a policy environment that, while empirically robust, may privilege economic and statistical reasoning while sidelining broader, more integrative perspectives that are also crucial for addressing complex societal challenges.

The Insight explores how empirical approaches can be complemented to better value the wisdom of interdisciplinary experts. It examines the implications for research and innovation policy and considers how complementary methodologies, such as policy narratives, could provide a more holistic foundation for policymaking while maintaining the empirical rigour that makes EBP valuable.

This Insight takes it as given that EBP has provided a significant advancement over previous paradigms and should remain central to the development of public policy.

The Essential Contribution of EBP to Policy Quality

Evidence-based policy has revolutionised governance by:

• Reducing the influence of ideological bias and political expediency in decision-making

• Providing clear metrics for policy evaluation and accountability

• Enabling more efficient allocation of limited public resources

• Facilitating cross-jurisdictional learning through comparable data

• Identifying causal relationships that inform more effective interventions

In research and innovation specifically, quantitative evidence has been crucial for demonstrating the relationship between R&D investments and economic growth, justifying public expenditure in this domain, and developing targeted interventions that maximise returns on innovation investments.

The economic and statistical methodologies that underpin EBP have created a more rigorous, transparent, and accountable policy environment—advances that should be preserved and strengthened. The critiques presented in this paper should be understood not as arguments against evidence-based approaches, but as suggestions for their enhancement through complementary frameworks.

Does Empiricism Marginalise Interdisciplinary Thinkers?

The Dominance of Economic and Statistical Methodologies

The processes of contemporary innovation policy development and review are heavily influenced by economics and social sciences, disciplines that prioritise quantitative metrics, cost-benefit analyses, and statistical modelling. This reliance can create an epistemological bias in which forms of knowledge that cannot be expressed in numerical terms—such as historical analysis, ethical reasoning, or interdisciplinary synthesis—are undervalued. Critics argue that this bias marginalises qualitative insights essential for understanding complex systems (Cairney, 2016).

Economists are particularly influential in policy formulation because their models provide governments with an ostensibly objective basis for decision-making. This has led to policies prioritising efficiency and measurable impact over more nuanced, context-dependent considerations. Such policies often avoid high-risk bets by steering clear of "picking winners" or imposing perceived financial burdens on taxpayers.

In the current innovation policy environment, AI can enhance research by enabling data-driven insights, pattern recognition, and predictive modelling. But when applied without sufficient theoretical grounding or methodological rigour, AI can introduce challenges such as data misinterpretation, biased analyses, or inaccuracies in economic modelling.

It is worth recalling that the neoclassical model is a static equilibrium framework focused on allocative efficiency, treating technological change and innovation as exogenous residuals. While Schumpeterian models offer a dynamic alternative by emphasising endogenous innovation through the effects of creative destruction, they face challenges in empirical validation.

For example, while trade data supports predictions about product displacement driven by innovation, assumptions about knowledge spillovers and institutional homogeneity remain contested.

Complementing Quantitative Metrics with Qualitative Insights

While data-driven analysis provides the essential foundation for effective policymaking, even the most sophisticated quantitative approaches benefit from complementary qualitative perspectives. Wisdom-oriented experts who engage in qualitative and interpretive thinking offer valuable dimensions that can enhance empirically-driven policies by providing context, meaning, and ethical frameworks that numbers alone cannot fully capture.

This complementary relationship is particularly relevant in contexts like Australia's innovation ecosystem, where business and academic leaders must balance demonstrable outcomes with long-term vision. Rather than replacing empirical measures, qualitative insights can contextualise data, identify blind spots in quantitative approaches, and integrate dimensions such as cultural factors, ethical considerations, and historical patterns that enrich empirical understanding.

Complex social systems exhibit emergent properties, contextual dependencies, and value dimensions that benefit from a combined approach—one that maintains the rigour of quantitative assessment while incorporating the interpretive depth that qualitative analysis provides. This integrative approach does not diminish the value of empiricism but enhances its application through a more comprehensive understanding.

The Challenge of Translating Wisdom into Evidence

Public intellectuals and thought leaders contribute through deep reflection, interdisciplinary synthesis, and historical and philosophical inquiry. Their insights often arise from accumulated wisdom rather than experimental design or statistical inference. However, policymaking structures that rely on randomised controlled trials, econometric modelling, or large-scale surveys inherently discount these forms of knowledge.

In research and innovation policy, there is ample quantitative evidence on R&D expenditures and their relationship to productivity growth, but broader perspectives on the societal role of innovation—such as its ethical, cultural, and political dimensions—are often missing.

Intellectuals and thought leaders such as Joseph Schumpeter, who provided a conceptual foundation for innovation studies, and Peter Drucker, who shaped management thinking, made their contributions based on integrative thinking rather than statistical analysis. Today, such figures would struggle to be heard in a policy environment that privileges measurable impact over intellectual foresight.

The Politicisation of Expertise and the Rise of Technocracy

It may also be the case that EBP undervalues intellectuals and thought leaders due to the increasing professionalisation and bureaucratisation of policymaking. In many governments, policy advisory bodies and think tanks are staffed by technical experts trained in specific methodologies. This has created a technocratic culture in which only certain forms of expertise—those that fit within predefined policy models—are considered legitimate.

Public intellectuals and many thought leaders, in contrast, often challenge prevailing assumptions and engage in discussions and debates that do not conform to policy orthodoxy. This can make them politically inconvenient, particularly in an era where governments seek certainty and risk aversion rather than disruptive new thinking. As a result, intellectuals are increasingly excluded from formal policy discussions unless they can align their arguments with existing models of evidence.

Implications for Research and Innovation

The drive for efficiency and measurable inputs, outputs, and impacts has fostered a form of empiricism that can stifle curiosity and idea generation—qualities essential for research and innovation. Policymakers under pressure to deliver quantifiable outcomes may reduce support for high-risk exploration or long-term research with uncertain payoffs. This environment encourages incremental improvements over transformative thinking.

Globally—and particularly in Australia—this trend has been linked to challenges in fostering a vibrant innovative culture. Critics argue that prioritising short-term results over bold experimentation undermines the potential for groundbreaking advancements. Complementary approaches like policy narratives could help bridge this gap by integrating quantitative data with qualitative insights, stakeholder experiences, and ethical considerations.

The Problem of Authenticity

There is a problem, however. "Thought Leadership" has become a buzzword, co-opted by corporate marketing, consultancy firms, lobby groups, and public relations strategists. The term now applies as much to CEOs promoting their companies as it does to intellectuals producing original and disruptive ideas. The rise of social media has exacerbated this, making it easier for individuals to brand themselves as "thought leaders" without necessarily contributing meaningful insights.

A key distinction must be made between:

1. Genuine wisdom-centered voices – those who contribute to knowledge through rigorous research, interdisciplinary and integrative synthesis, and long-term engagement with complex ideas.

2. Corporate and lobbyist influencers and self-styled "thought leaders" - individuals who package existing selective knowledge into digestible soundbites, often prioritising personal branding over substantive intellectual contributions.

This differentiation is important because, without it, arguments for integrating deeper thinking into policymaking risk inviting a flood of superficial ideas driven by media visibility rather than intellectual depth.

The rise of a market for ideas—where influence is determined by visibility rather than intellectual merit—has led to several distortions:

1. Algorithmic Amplification: Social media platforms reward virality, not depth. This creates incentives for simplistic, sensationalised arguments rather than nuanced, evidence-based thinking. Voices that thrive in this environment may not be the most rigorous or insightful.

2. Corporate Capture: Many high-profile commentators are funded by industry, raising concerns about the independence of their ideas. Consultants and executives promoting particular narratives on innovation, digital transformation, or sustainability may have vested interests that align more with corporate strategy than public policy needs.

3. Ephemeral Influence: The speed of digital discourse means that many self-styled "thought leaders" achieve brief moments of prominence before fading into obscurity. This contrasts with the enduring intellectual contributions of figures whose ideas shape policy over decades.

The danger for policymaking is that governments, in seeking broader perspectives, may end up engaging with those who are most visible rather than most insightful. This can lead to policies being shaped by fads rather than foundational intellectual contributions.

Balancing Rigour with Broader Perspectives

Given these challenges, how can policymakers integrate substantive thinking without falling into the trap of public relations spin? Several strategies offer promising directions:

• Carefully curating intellectual voices. Policymakers should distinguish between substantive intellectual contributions and self-promotional commentary. This means prioritising scholars, practitioners, and interdisciplinary thinkers with a track record of serious engagement rather than those who thrive on personal branding.

• Requiring transparency. Scholars and visionaries engaging in policy should disclose financial relationships and conflicts of interest, much as researchers are required to do. This would help separate independent thinkers from those advancing corporate or ideological agendas.

• Reinforcing institutional gatekeeping. Universities, think tanks, and policy institutes have historically legitimised intellectual contributions. Strengthening these institutions as validators of expertise could counter the marketisation of intellectual leadership.

• Governments should value intellectual depth over popularity. They should engage with thinkers whose work demonstrates long-term intellectual depth rather than those who simply attract media attention. This requires shifting from reactive policymaking (based on trending ideas) to proactive engagement with foundational thinkers.

Towards a More Inclusive Policy Framework: The Role of Policy Narrative

One way to rebalance policymaking is to embrace policy narrative as a complementary approach to evidence-based policy. Policy narratives recognise that policy decisions are not made in a vacuum but are shaped by historical contexts, cultural values, and moral considerations. They integrate qualitative insights with empirical evidence, allowing for a richer, more multidimensional understanding of complex issues.

A policy narrative approach would involve:

• Integrating Historical and Philosophical Perspectives – Recognising that past intellectual traditions can offer valuable lessons for contemporary policy challenges.

• Embracing Qualitative and Interpretive Methods – Valuing case studies, oral histories, and ethnographic research alongside quantitative analysis.

• Encouraging Interdisciplinary Dialogue – Bringing together economists, historians, sociologists, and contemplative scholars to formulate more holistic policies.

• Prioritising Public Engagement – Ensuring that policy debates are not confined to technocratic circles but involve broader societal participation.

By adopting a more narrative-driven approach, policymakers can make better use of the insights offered by interdisciplinary thinkers, ensuring that policy decisions are not only empirically sound but also ethically, historically, and socially informed.

Conclusion: Strengthening Evidence-Based Policy Through Integration

Evidence-based policy has transformed policymaking by ensuring decisions are guided by rigorous analysis and empirical data—an advancement that should be preserved and extended. The argument presented in this paper is not that EBP should be replaced or diminished, but rather that its strengths can be enhanced by thoughtfully integrating complementary approaches that capture dimensions of complex problems that quantitative metrics alone might not fully address.

By supplementing the robust empirical foundation of EBP with insights from interdisciplinary thinkers whose contributions draw on historical perspective, ethical reasoning, and systems thinking, policymakers can develop more comprehensive approaches to research and innovation challenges. This integration addresses potential gaps in current approaches without sacrificing the rigor and accountability that make evidence-based policy so valuable.

A balanced approach that maintains the centrality of empirical evidence while incorporating policy narratives would create a more resilient policy framework—one that harnesses both the power of data and the wisdom of broader perspectives. This is particularly important in research and innovation policy, where transformative breakthroughs often emerge from intersections between disciplinary boundaries and require both quantitative assessment and qualitative understanding.

While we must be cautious about how broader perspectives are incorporated—distinguishing between serious intellectual contributions and public relations-driven commentary—this careful integration promises a more holistic approach to policymaking that builds upon, rather than replaces, the empirical foundation that EBP provides.

The future of effective policy development lies not in choosing between evidence and wisdom, but in their thoughtful integration—maintaining empirical rigor while expanding our conception of what constitutes valuable input to the policymaking process.

References

Cairney, P. (2016). The Politics of Evidence-Based Policy Making. London Palgrave Macmillan.

Dodgson, M., Gann, D., & Salter, A. (2005). Think, Play, Do. OUP Oxford.

Drezner, D. W. (2017). The ideas industry: How pessimists, partisans, and plutocrats are transforming the marketplace of ideas. Oxford University Press.

Kurtzman, J. (1998). Thought leaders: insights on the future of business. Jossey-Bass Publishers.

Nichols, T. (2017). The Death of Expertise: The Campaign against Established Knowledge and Why It Matters. Oxford University Press.

The Acton Institute for Policy Research and Innovation advances public policy through evidence-based research and expert analysis, focusing on science, technology, and innovation. Led by Dr John Howard, the Institute delivers practical policy solutions and fosters learning and adaptation to improve public outcomes in Australia’s innovation system