Dropping Old Theories about Innovation
Innovation is a complex process, but similarities exist in all innovative efforts. Existing innovation theories propose that linear, progressive and random forces determine the direction of innovation. Now, advances in dynamic systems theory, a branch of mathematics, enable a more intense study of chaos. Studying the development of innovation through this prism shows that innovation is nonlinear and cyclical. The innovation process repeats cycles of divergent (scattered) and convergent (unified) activities over time and at different levels in your organization.
“Innovation managers are to go with the flow - although we can learn to maneuver the innovation journey, we cannot control it.”
You must throw out previous thinking about innovation. The discards include theories that innovation happens in organized progression; that innovation is affected by random forces; that innovations converge to a common outcome regardless of how they originated, and that the whole process has a degree of predictability.
“Whereas invention is the creation of a new idea, innovation is more encompassing and includes the process of developing and implementing a new idea.”
Recurring cycles of divergent and convergent activity can be seen in several major factors that influence innovation, including the development of economic structures that support innovation, the manner in which learning occurs within innovation teams, the ways that management is organized, and how managers behave and cooperate with other organizations. The innovation journey is not stable and predictable, and it is not stochastic. Innovation is far more complex and sensitive to external influences than previously thought. This discovery has implications for innovators and managers. Old beliefs to discard in favor of new findings include:
- Old: Innovation is the implementation of one idea.
- New: Multiple ideas are involved. Some ideas are implemented and some are discarded.
- Old: One entrepreneur or group of entrepreneurs takes an idea from beginning to end.
- New: An innovation team consists of a changing group of entrepreneurs who fill a variety of roles. The innovation journey is an accumulation of multiple actions by multiple people. Long gestation periods and multiple stops and starts characterize innovation.
- Old: Implementing an innovation heralds the arrival of a stable new order.
- New: An innovation usually generates many spin-offs and related projects as it is integrated into the old order.
- Old: Innovation proceeds in progressive stages or phases of development.
- New: Innovation follows both divergent and convergent paths. Some of these paths turn out to be related to the final outcome of the innovation and some do not.
- Old: An innovation is assessed when the final product is rolled out.
- New: Innovations are constantly evaluated during their entire innovation process.
“The focus on people as creators and facilitators of innovation needs to be balanced by equivalent attention to people as inhibitors of information.”
Innovation can be technical or administrative - building better nuts and bolts or finding better ways to shuffle paper. But mechanical and technical innovation cannot happen without the support of an innovative administration. The innovation manager must understand the dependent relationship between technical and administrative factors.
The Process
Conducting in-depth, multi-year case studies on three innovative products or services, researchers examined, recorded, and analyzed every detail, start, stop, failure, and success. The core processes of innovation were similar in all three. Any differences in them could be attributed to the individual organization’s setting and structure.
“It is primarily through repeated trials and the accumulation of learning experiences across these trials that an organization can build an inventory of competence and progressively increase its odds of innovation success.”
Initially, the innovation journey takes similar paths, regardless of the organization, the idea, or the organizational forces that promote the idea. Innovation is spurred by shocks in the organizational structure. At the outset of an innovative process, the idea becomes scattered among divergent groups. Some are on similar paths and some are on divergent paths, but their multiple activities spur creativity.
“Understanding the close connection between technical and administrative dimensions of innovation ideas is a key part of understanding the management of innovation.”
In this process, wrong turns and missteps occur frequently. These mistakes can be caused by plans that don’t work out or by unplanned environmental forces. Learning and adaptation must occur so that an innovation can continue despite failure. Such failure often moves an innovation in new or different directions.
“Innovation is a network-building effort that centers on the development of transactions or relationships among people who become sufficiently committed to their ideas to carry them to acceptance and legitimacy.”
End users must have input in the innovation process if you want them to accept the innovation. When end-users have little or no input, innovation is inhibited. For ideas to be developed and adopted in a timely way, managers, innovation teams, and end users must communicate. Management plays a crucial role throughout this journey. Although it cannot guarantee success, sensitive, flexible and experienced innovation management can increase the odds of success. Experience with multiple, complex innovations also increases an organization’s chance of success with a successive innovation. Innovation is a learning game, and the more you know, the better you go.
“Another common characteristic of the innovation journey is that setbacks frequently arise because initial plans go awry or unanticipated environmental events occur that significantly alter the ground assumptions and context of the innovation.”
Managers must evaluate an innovation’s progress during its journey to see what is and what is not working. As a manager, you need to establish criteria to gauge the relative merit of every action. However, these criteria must be as flexible as the overall innovation process, and must change in response to new ideas, successes and failures. Definitions of success and failure will also change frequently, and these changes will affect the direction and outcome of innovation. Learning becomes more difficult when definitions of success and failure shift.
Learning About Innovation
Learning by testing is decision rationality. Learning by discovery - or through actions taken without clearly defined goals - is action rationality, which is especially significant in the early stages of innovation. Conflicting goals among team members, and between team members and managers, complicate the process, but often lead to new discoveries.
“Negative outcomes will trigger interventions from external resource controllers, and these interventions may subsequently lead to changes in the course of action being pursued by the innovation team.”
Outcomes are innovation’s end product - the results of successfully realized ideas. Yet, outcomes can also be negative by-products of innovation - missteps, failures, screw-ups, funding cuts and disasters. These results are not really negative, but are simply steps along the way, and opportunities to learn and grow. Mistakes ultimately can contribute to the success of a project, because negative outcomes lead to changes in goals and criteria.
Divergent and Convergent Activity
Managers must understand the idea of divergent and convergent activity, so they can guide projects through either kind. Ambidextrous management skills are necessary because the landscape of innovation can change quickly from periods of convergent activity to periods of unpredictable divergent activity.
“Although innovation journeys can follow many different paths and outcomes, the underlying pattern is remarkably alike.”
Divergent activity expands in different directions. You can spur it by adding resources to a project. Divergent activity is the creation of ideas through inspiration and negotiation, random or chaotic patterns, learning by discovery, encouraging and balancing different views, building relationships and networks and creating an infrastructure for competitive advantage. You may have to deal with this chaos by using your intuition.
“The institutional leader role is important at the earliest stages of an innovation, as the innovation team forms, and later when an innovation moves from development to implementation to market introduction.”
Convergent activity is characterized by integration and a single focus, specific ideas and strategies, trial-and-error learning, unity and consensus, and an operating infrastructure designed to gain competitive advantage. You can manage convergent situations rationally.
The Human Variable
People are a significant variable in innovation. The people working on a project are involved in an inherently creative process, so any actions they take and any outcomes, whether positive or negative, reflect the personalities involved and the relationships among them. As a manager, you must meet the challenge of efficiently organizing and utilizing personalities.
Top managers who are directly involved with the innovation process tend to play different roles - institutional leaders, sponsors, mentors, and critics. Different managers involved with the same project usually make decisions independently, and those independent decisions serve to check the direction of other managers. Pragmatic management has a significant impact on your team members’ emotions and behavior, so management’s flexibility is especially important when the team faces negative outcomes.
It’s remarkable that the innovation process happens at all. Conflicts occur at every level and at every stage - between innovators and groups, between managers and innovators, between managers and other managers, and between managers and top management. Bilateral relationships develop between organizations and people in independent, complex, and unpredictable ways. These relationships work at periods of high and low intensity at different points in the process, and spill over onto other relationships. A punctuated equilibrium is at work in this collaborative web of relationships. During periods of unstable, divergent activity, partners negotiate and change their function and direction. During the longer, relatively stable periods of convergent activity, groups in the different organizations know what they must do and do it. When the bilateral relationships become dependent on input from other organizations, the web of interdependent relationships becomes a network of relationships. At this point of self-organizing criticality, the web overtakes the bilateral relationship as the most important structure supporting innovation.
Infrastructure and Partnership
Capitalism has remarkable regenerative energy - constantly tearing down, building anew, reworking and reinventing. The innovation process is the spark that lights the capitalist fire. But all these wonderful new elements must be built on a framework, the complex infrastructure of innovation. A critical combination of public and private resources creates the political, economic and market infrastructure that encourages innovation. Key initial infrastructure for innovation includes institutional norms, scientific knowledge, financing mechanisms, competent human resources and informed and sufficiently affluent consumers. Government must partner with private concerns to establish a framework that encourages innovation. Government also must create and enforce mechanisms to regulate the behavior of competing firms (think U.S. vs. Microsoft here) and to legitimize the actions of private entities in the larger industrial, political, and social culture.
Firms must cooperate to maintain the infrastructure that encourages innovation, but at the same time they must compete for market share, public resources and good workers. Each firm strives to find an individual market niche, but cooperation is sometimes necessary. This competition vs. cooperation balance is a paradox in the innovation process. The need for private firms to cooperate with public regulatory agencies, even when those agencies’ decisions could hurt them, is also paradoxical. These conflicts reinforce the idea that innovation and creativity arise from struggle and conflict - but on a bigger scale.