Community Indicators and Community Learning: An Exploration
President, Adaptive Communities, Lemont, PA USA
Senior Associate, Community Initiatives, Inc., Boulder, CO USA
Do not cite or quote without permission
Indicators are pieces of information that reflect some larger system (although that "system" is often not explicitly considered or defined). Their primary function is to communicate something about that system to an audience (or audiences) that are unlikely or unable (for a variety of reasons) to spend significant amounts of time seeking comprehensive, detailed information about that system. The number of communities seeking to develop and use indicators has grown significantly over the past decade, becoming a core tool for hundreds of community improvement processes across the country (and world) and cutting across sustainability, quality of life, healthy communities, and other frameworks for organizing improvement efforts. At least one thing is common across these various efforts - people presume that the development and reporting of indicators will help create positive change within their community.
For the past ten years, I have been working with a wide variety of places, organizations and people to develop community indicators, performance measurement systems, and information-based collaborative processes to effect change. This paper, while drawing upon the work, research and writings of others, is primarily an attempt to reflect upon and synthesize my experiences and observations, and to offer some propositions regarding how to more effectively conceive of and integrate indicators within a larger framework for building healthier, resilient and more sustainable communities.
Most people still seem to approach community information systems as decision-support tools, the primary purpose of which is to bring information to decision makers in ways that directly inform particular decisions. For example, I have seen many people and projects seeking to develop indicators as a way to provide answers. Rarely, however, is there a case where participants in these processes can point to a specific indicator or piece of information and say with confidence "I changed my decision based upon that."
Throughout my work, however, I have observed participants in these community efforts
- Explore and create a better understanding among participants about the different legitimate ways of viewing or defining "the system" and how the system "works";
- Identify and come to agreement upon the various valued outcomes, goals, visions community members have with regards to the system;
- Engage each other in discussions about what information should be collected and reported;
- Become more informed about where data/information resides, who collects it and why, and how to access and interpret that information;
- Come together to discuss what that information means for the community;
- Develop new relationships and networks;
- Develop new understanding among community members of each other and key issues affecting the community; and
- Discover new abilities among individual citizens and community organizations.
It is this "learning and capacity-building" effect of indicator development and use, often viewed as "secondary" or intangible, that I believe to be the most significant and potentially powerful effect of community information systems. From the simultaneous development of shared meaning and stronger networks of caring and trust, action has emerged and been sustained in ways nobody in the project would have predicted.
This paper will explore some of the emerging lessons regarding how indicators can be effective for promoting positive change in communities. Following a brief review of these lessons, it will review of the characteristics of communities that reflect and learn as they engage in community development/action, and then consider how the development and use of community indicators can be part a larger system for "community learning."
II. What Makes Indicators Effective?
Why have communities, regions, organizations and networks expended enormous quantities of financial and human resources to develop sets of indicators? While indicators can be used for a wide range of purposes, the most basic and universal answer is simple - "They know they want the indicators to become part of a public dialogue and somehow to help communities and regions become better at self-management and more self-conscious about the direction they are going. They want the indicators to be influential."
Over the past few years, there have been a number of gatherings of indicator practitioners aimed in part at identifying the various elements of both indicator development processes and products that contribute to positive results. From my review and synthesis of the results from these gatherings, there appears to be some convergence among practitioners upon a set of emerging propositions regarding when, how, and what makes indicators effective. In general, these propositions tend to cluster around three aspects: (a) the processes for selecting and developing the indicators, (b) characteristics of the indicators themselves, and (c) the systems and processes for using the indicators.
Indicators are more likely to be used and have an impact on community decisions if, within and through the process of selecting and developing the indicators:
- Participants develop agreement on:
- System scope and boundaries
- Project purpose, audiences, & desired outcomes
- The indicators are linked to desired outcomes (i.e., community vision and goals). And, if these outcomes do not already exist, community members take the time to articulate a vision or set of goals for their community.
- Both the intended users as well as those with a "technical" knowledge of the system are engaged in the selection of the indicators.
- Project leadership or conveners view the process as a key opportunity to build capacity and social capital among community members (i.e., build trust and learning through the process).
- Individual & organizational (government, business, non-profit) "champions" - change catalysts and implementers - are identified, developed and nurtured.
- Opportunities are taken to engage and learn from organizations & agencies with outcome-based management experience.
Indicators are more likely to be used and have an impact on community decisions if, the indicator system includes:
- Indicators for important elements or relationships of the system, as identified and defined by community members and, specifically, the intended users
- A mix of indicator "types," including those that help identify "root causes" of problems.
- Indicators that are clear and easy to understand or interpret by their intended audience.
- Measures that are feasible (i.e., based upon data currently available), but with the understanding that they will be improved over time
- Sets of indicators that enable users to make comparisons over time and/or across places, and associated narratives that make the implications or "stories" more transparent.
And finally, indicators are more likely to be used and have an impact on community decisions if, within and through the process of using the indicators:
- Project leadership find or provide resources for specific planned opportunities, processes, or events for the intended audience to use the indicators and provide feedback on their relevance and utility (i.e., don't assume that if you report them they will be used).
- They are embedded within or linked to existing decision & planning processes. Rather than having general, non-specific discussions regarding what the indicators say and what should be done, they are part of a more explicit and formalized "plan-do-check-adapt" cycle with links to goals, targets, and specific decisions or policies.
- Different reporting formats and processes are used which are appropriate to the particular needs and timing of the intended users.
- Networks of local organizations that are developing & using measures are created or nurtured so that they can share learning and resources and pursue opportunities for collaboration.
- Project and community leadership understand role that indicators can help create change, but that other elements and capacities also need to be in place; indicators are but one piece of a larger "change model" for community improvement.
While these are all reasonable propositions, they remain aimed primarily at the level of practice. This not meant as a criticism, but rather as an observation that they need to be complemented by some larger discussion of how indicators "fit" with and support different models for promoting community efficacy and change.
As noted by two observers of past and present efforts to develop indicators, "these efforts . . . have relied on unrealistic expectations and a simplistic model of how information drives policy and public action." Noting that indicators can and do have influence on decision-making, Innes and Booher suggest that:
The influence came through a much more complex and less observable process than many recognize . . . . Indeed, it was not really the indicators themselves or the reports that mattered, but the learning and change that took place during the course of their development and the way the learning led to new shared meanings and changed discourses. This learning and changes in practices, however, was highly contingent on the way information was developed and who was involved.
Innes and Booher go on to describe eight lessons from past research and review of indicator efforts that describe when and how indicators have been influential, including:
- Indicators do not drive policy, but rather influence it through a process of conversion and learning;
- Indicators primary impact occurs through and during the process of developing and discussing them;
- Their influence is felt most through a "collaborative learning process" as those who develop and use them jointly make sense of why the indicators are important, what they mean, and their implications for changes in actions and policies;
- The full range of anticipated users (i.e., those whose decisions one hopes will be influenced by the indicators) must be involved in the selection, development of and collaborative discussions regarding the meaning of the indicators.
Their focus on indicators as catalysts for collective learning leading to collaborative action is consistent with the more specific propositions emerging from practitioners. It also provides a link to another body of work looking at the capacities of communities (and their members) that enable them to learn and adapt as they engage in efforts to improve their well-being. It is this notion of "community learning" that provides some interesting possibilities for considering how indicators can and do help create stronger, healthier, and more resilient communities.
III. Communities That Learn
"Learning is a complex process that goes beyond simple acquisition or creation of new knowledge and skills. Newman (1999, p. 85) suggests that learning has a transformative aspect, which has to do with understanding values, ideas and pressure from peers that constrain the way we think and act. Learning interactions take place between individuals, sometimes mediated by text or other media. Networks enable people within a community to come together to share their values and interests (Lane & Dorfman 1997) just as networks operate at regional levels to allow collective learning."
Community Learning and Sustainability (2000)
"An institution that learns while it acts - a reflective institution - will necessarily look different from an institution designed solely to act. In a reflective institution, monitoring and evaluation of activities and projects is not so much a discrete task as a way of thinking which must permeate the structure, philosophy and practices of an institution."
Reflective Institutions (IUCN, 1997)
The idea of a "learning organization" has been with us for long enough now that it has become part of the normal set of terms we use to discuss the characteristics and capacities of effective, high-performing private sector companies and public sector agencies. Over the past five years, a few scholars and practitioners have begun to combine the findings and propositions from work in the areas of learning organizations, community capacity, social capital, adult education and adaptive management, and suggest how these have particular relevance for considering why and how some communities seem to be more effective and "high-performing" than others. Specifically, they have begun to identify an emerging set of qualities and characteristics of communities that learn and, as a result, are more effective at adapting to external forces and in shaping their future development path.
Based upon an initial review and synthesis of this work, I have identified nine characteristics of a community that learns, including:
- Admitting incomplete knowledge
- Value diversity, engage the whole community.
- Two-way feedback systems
- Effective networks
If one assumes that s/he already knows the answer or has the best approach, or that the "experts" responsible for crafting community policies and plans know how best do address community needs and achieve community goals, learning is unlikely to occur. (Or, the barriers to learning have been raised significantly.) However, when we admit that our communities are very complex systems and that none of us really know the most effective way for achieving our collective well-being, we open ourselves to a new approach to planning and action. Hypotheses within project and strategic planning are made explicit. Plans and strategies are seen as opportunities to test these hypotheses, and to improve community knowledge that can be used to improve future choices and actions.
Once we admit that we do not have all the answers, and that all of our decisions can affect the future of our community, then it becomes more apparent that "[e]verybody has a role in the learning and wisdom generating process." Communities that learn create and sustain processes which engage a broad and diverse range of community members, recognizing that every member is both a source of knowledge as well as a potential decision maker who will affect the community's future. "They seek new information and different ideas to add to the local mix of experience and wisdom" and "recognize everyone for their contribution."
For learning and change to occur, it is critical that there is a continuous and valued two-way flow of information between community members and community leaders (both formal and informal). Noting that within communities there are thousands of actors whose decisions can and do effect its character and conditions - past, present and future - Innes and Booher suggest that we use not the analogy of a machine but rather an organism "which evolves and changes its direction in response to external events and to its own internal dynamics." Rather than attempt to direct change through a top-down or comprehensive planning intervention, they propose another approach:
Such a system is capable, however, of improving itself in ways we could not predict, but which are more effective that what the most sophisticated analysts could create. Such self-improvement and adaptation however requires feedback - various kinds of information - to flow among the players who make the city what it is.
There is much evidence that the presence and strength of networks within communities is a key and perhaps necessary ingredient for effective communities. In the case of community learning, three types of networks all play important roles:
- "Strong ties" among group members (Intra-community)
- "Bridging ties" between groups and/or between communities
- "Linking ties" between public and private institutions
The presence of all three types of networks facilitates the flow of information, resources and the formation of a sense of connectedness and reciprocity among community members that enable collaborative learning. In addition, they enhance access to the wide range of internal resources available to a community through its various community members, local government and private organizations, as well as the resources (knowledge, $, etc.) available from other "surrounding" communities. As noted by Kilpatrick:
The presence of bridging (or 'weak') ties between groups within a community and between communities, and linking ties with public and private institutions, in addition to bonding ties [i.e., intra-community networks], has a positive impact on community sustainability. The right mix of the three kinds of ties strengthens the social capital of the community by giving it an external dimension. This enables the community to deal with internal and external problems or changes through access to a wide range of internal and external knowledge, skills and resources.
The community has articulated an explicit vision or set of desired outcomes for their future that continues to be referred to and developed by members as they act. Without such a collectively articulated vision, there is no foundation or point of reference for people and organizations to come together to answer the question "are we better off than we were before?" Inherently, the discussion this question initiates starts with another set of questions: "What does better off mean for us? What do we want are community to be, look and feel like in the future?" While individual organizations and community members can and do learn as they engage in their own improvement efforts, efforts to move from individual to community-level learning requires some agreement on answers to these types of questions.
Community members seek to understand the interconnections between their interests, issues and activities and the larger community-level systems and desired outcomes (i.e., "the big picture"), but not at the expense of identifying and taking effective actions that address specific issues or problems. Community learning occurs as people take action and then collectively reflect upon how those actions impact desired outcomes. As we often see in collaborative processes, if the focus remains on the desired outcomes, there results a wonderful vision for the community but with no activity to achieve it (and no "text" for learning). Likewise, if there is only action with no reference back to whether community health and well-being is being improved, there is a lot of heat being generated but not necessarily being translated to energy that is moving the community in the desired direction.
In a recent evaluation of an effort to improve the capacity of non-profit organizations through the development and use of performance measurement systems, the reviewers concluded that "establishing these systems alone was not enough. In the end, the project's success had less to do with whether measurement systems were developed and more to do with whether the organizations were able to create a culture that valued the process of self-evaluation." The same can be said for communities as they attempt to develop data warehouses and sophisticated systems for measuring performance. Unless learning and reflection occurs throughout the community - and not just as a "special planning project" or as the responsibility of one agency - the use of the data will be limited as will the resulting learning. As they take action, individuals and organizations across the community need to monitor their own results and share lessons with other community members. Learning from action is valued and rewarded. "Good or bad, learning communities share [and systematically review] the results of projects, actions, and events."
As Meg Wheatley notes, "Thinking is the place where intelligent actions begin." Yet, both individually and as a society, we are speeding up our processes and giving ourselves less not more time to think and reflect. Learning - individual, group and community - requires that spaces for reflection exist and that institutions and community members reclaim the necessary time to talk, reflect and share their experiences. "Discussion, dialogue, conflict, and reflection are part of the learning process." Yet each one of these processes, if they are to be positive community-building experiences that facilitate collaborative learning, often must move at a slower pace and within longer timeframes than our current decision-making culture supports.
"If we feel we're changing in ways we don't like, or seeing things in the world that make us feel sorrowful, then we need time to think about this. We need time to think about what we might do and where we might start to change things. We need time to develop clarity and courage. If we want our world to be different, our first act needs to be reclaiming time to think. Nothing will change for the better until we do that."
An essential component of community (as opposed to individual) learning is "the capacity of individuals to come together and share their knowledge and skills to solve local problems. Partnerships and collaboration in communities mean a wider range of skills are acquired by people, and this enhances community capacity to manage change." If people and groups are unable to bridge their diverse perspectives and experiences, then it becomes more difficult to imagine how the communication and joint action necessary for community learning will occur.
IV. Indicators and Community Learning
What is the relationship between developing and using indicators and building the capacity for community learning? Perhaps there does not need to be any. Certainly a community could develop indicators without explicitly considering or building a fuller system for community learning. Likewise, a community could seek to build many of the above capacities associated with community learning without developing an indicator system.
The synergy of the two systems, and activities to build those systems, however, should not be ignored. In the case of indicators, it appears that they have the greatest effect where they are approached as tools for effectively raising questions, providing the basis for people to deliberate, and to help communities come together to develop shared meaning about important issues. Clearly, to realize their full potential the indicators are just one element of a larger set of capacities for community change. They are one piece of a community learning system.
As we consider the nine elements above that begin to describe a learning community, an indicator system can play at least one direct role - as a central part of a community feedback systems (#3). At the simplest level, general outcome indicators (those most often found in community-level reports) offer an initial set of feedback to community members regarding whether or not their community is moving towards or away from their vision of a desired future. Increasingly, we are seeing communities develop more sophisticated and detailed indicator systems with measures that link back not only to community-level outcomes, but also neighborhood conditions, program or organizational outcomes, and even information that can be used in very short timeframes by individual decision makers.
However, as we have noted throughout this paper, the value and benefits of a system of indicators is often realized not so much from the simple presence of the indicators, but through the processes for developing and using them. For example,
- as participants in a community indicator effort develop agreement on what the indicators should be and what sets of decisions they hope the indicators will inform, the discussions can help surface the partial knowledge each of us have (#1), develop capacities for collaboration (#9), and begin the process of taking time for reflection (#8);
- as community indicator efforts seek to engage and learn from community members and local organizations with relevant experiences in outcome measurement, new networks are developed which broaden the flow of resources and information into and through the community (#4);
- as indicators are used and linked to specific planning and decision processes through some form of "plan-do-check-adapt" cycle, this encourages a culture of inquiry (#8), exposes incomplete knowledge regarding cause and effect relationships (# 1), and reinforces connections between specific actions and community outcomes (#6).
As we look at each of the "best practices" for indicator development and use (Section II, above), we can identify how they support one or more of the nine characteristics of community learning. Similarly, we can go through the inverse exercise and see how each of the nine community learning capacities facilitates more effective development and use of indicators.
V. Pulling It Together (or, "Where Do We Go From Here?")
As noted in the beginning, the primary function of indicators is to communicate information about complex systems (in this case, community health or sustainability) in order to promote decisions and actions that will improve those systems. As outlined above, it appears that a primary mechanism for this is through the learning - both individual and collective - that occurs as community members engage with one another in the various discussions and tasks necessary for selecting, building and using the indicators. While the various characteristics of communities that learn suggest the capacities necessary for community indicators to have their full effect as tools for community improvement, they do not lead us to any particular change process or model.
A recently released report from the Rockefeller Foundation may provide another piece to the puzzle. In Communicating for Social Change: An Integrated Model for Measuring the Process and Its Outcomes, Figueroa et al offer the Integrated Model of Communication for Social Change (IMCFSC) which "describes an iterative process where 'community dialogue' and 'collective action' work together to produce social change in a community that improves the health and welfare of all of its members." The model includes three main elements:
- The "catalyst" - some event, person, or information that in turn leads to
- Community dialogue and collective action producing
- Individual and social change.
Within each one of these broader elements, the authors describe a range of steps and activities that can occur to move people and communities through the model (see Figure 1 below).
Without going into a fuller discussion of the model, it seems to suggest at least three potentially useful areas for further consideration and discussion among indicator and community change practitioners.
First, whereas the nine capacities for community learning provide a "structural" description, the IMCFSC begins to provide a dynamic flow of events and activities leading to individual and social change. While the concept of "learning" is not specifically addressed or used within the IMCFSC, it seems not a step too far both to link the capacities for community learning as the characteristics that facilitate movement through the IMCFSC and to view the mechanism by which all of this occurs as social interactions that lead to learning - individual and social/community.
Second, the role of the catalyst is critical in initiating the flow of activities through the model. While their discussion of how some "stimulus" actually triggers the "community dialogue - collective action" phase is relatively brief, it is not hard to imagine (or observe in many communities across the country!) community indicators as a catalyst for community discussion. "Once this discussion is initiated it may unfold in several directions: from simply creating a greater sense of dissatisfaction, to inciting a community conflict or to cooperative action that helps solve the problem." It seems the key question is how and when do indicators as a catalyst lead down the latter path (i.e., cooperative action). I would suggest as an initial proposition that the presence of the nine elements for learning communities would make it more likely that "indicators-as-catalysts" would lead to cooperative action.
Third, the IMCFSC helps to focus our attention on the necessary synergy between individual and social change (or, to use different terminology, individual and collective learning). As Figueroa et al observe, many individual behavior change programs in the public health field typically are designed to achieve outcomes associated with a single, specific aspect of health (e.g., condom promotion for HIV/AIDS prevention). "As a consequence, some individual behavior change may even be limited to a short duration in time unless other measures are taken to ensure that such changes are institutionalized or self-sustaining." On the other hand, if the focus is only on "social change", the capacity or potential for improvement may increase but there may be little or no actual changes in the health and well-being of individuals. In short, what is needed for sustained change is both individual and collective learning.
It has been the intent of this paper to review and reflect upon emerging lessons regarding how and when community indicators can be effective tools for community improvement. In doing so, it seemed necessary to look more broadly than just the community indicators field itself and into broader discussions regarding social communication, feedback and learning as a mechanism for change, and capacities for community learning.
While it is not the intent of this paper to reach conclusions, there does appear to be at least one theme worth including as a concluding proposition. Given the complexity of how communities function and the uncertainty regarding cause-and-effect relationships in the problems they face, we need to design flexible, resilient systems that seek to engage broader sets of people (a) in sharing and making collective meaning with their knowledge and perspectives, (b) in generating shared hypotheses or "best practices" regarding community improvement strategies, (c) reflecting on and learning from the results of those strategies and (d) adjusting their plans, strategies and practices. While indicators are fundamental components of these systems, we must look to create and sustain the broader sets of capacities for community learning if indicators are to be effective tools for community improvement.
From Maria Elena Figueroa, D. Lawrence Kincaid, Manju Rani, Gary Lewis; Communication for Social Change: An Integrated Model for Measuring the Process and Its Outcomes, Rockefeller Foundation (2002)
- Innes, J. E. and Booher, D.E. Indicators for Sustainable Communities: A Strategy Building on Complexity Theory and Distributed Intelligence. Working Paper 99-04. Institute of Urban and Regional Development, University of California at Berkeley. Sept. 1999, p. 5.
- - Proceedings of the Colorado Forum on National and Community Indicators (November 22-23, 1996); Community-Based Information and Sustainable Community Development: Symposium Findings & Final Report, Green Mountain Institute (October 17-19, 1999); Rocky Mountain Institute Indicators Workshop Proceedings, Rocky Mountain Institute (May 2-4, 2001); Doing and Measuring: Proceedings from the 2000 State-of-the-Fraser Basin Conference (November 24-25, 2000); Proceedings of the California Community Indicators Conference, Redefining Progress (December 3-5, 1998).
- Innes and Booher (Sept. 1999), p. 6.
- Innes and Booher, p. 6
- Innes and Booher are not the only authors who have sought to review and comment upon those characteristics of past indicator efforts which seem to lead to greater impact. For an introduction to and review of the wider range of literature, see Randa Gahin and Chris Paterson, "Community Indicators: Past, Present and Future," National Civic Review v. 90 (Winter 2001), pp. 347-361.
- This paper will not seek to discuss the history nor principles embedded within the idea of a learning organization. Although many people have worked to develop this idea and its translation to practice, the early work of Peter Senge (The Fifth Discipline, 1990) remains the foundation for the field and is the referent and point of departure for this discussion.
- The following synthesis is based upon a review of the following:
Also consulted were "Presenting Community-Level Data in an 'Outcomes and Indicators' Framework: Lessons from Vermont's Experience." David A. Murphey. Public Administration Review, v. 59 (Jan/Feb), 1999; "A Community Indicators Case Study: Addressing the Quality of Life in Two Communities," Kate Besleme, Elisa Maser, & Judith Silverstein (Redefining Progress. March 1999); "Neighborhood Indicators: Taking Advantage of the New Potential," G. Thomas Kingsley (National Neighborhood Indicators Partnership, The Urban Institute. October 1998).
- Ian Falk and Lesley Harrison, "Community Learning and Social Capital: 'just having a little chat'," Journal of Vocational Education and Training, Vol 50 (1998): 609-627;
- Sue Kilpatrick, "Community Learning and Sustainability: Practice and Policy," Centre for Research and Learning in Regional Australia Discussion Paper D6/2000, University of Tasmania (2000);
- Allen B. Moore and Rusty Brooks, "Learning Communities and Community Development: Describing the Process," Learning Communities: International Journal of Adult and Vocational Learning, Issue 1 (November 2000); 1-15;
- Ron Faris and Wayne Peterson, "Learning-Based Community Development: Lessons Learned for British Columbia," Report to the Ministry of Community Development, Cooperatives and Volunteers, Province of British Columbia, Canada (July 2000);
- Eric Dudley and Alejandro Imbach, Reflective Institutions: Eight Characteristics of Institutions that Encourage and Respond to Learning by Doing," International Union for the Conservation of Nature and Natural Resources (1997), An Approach to Assessing Progress Toward Sustainability - Tools and Training Series;
- Michael Gurstein, "Community Learning, Community Economic Development and the New Economy (DRAFT)," Report to the Community Learning Networks Secretariat, Office of Learning Technologies, Human Resources Development Canada (2000);
- Shanna Ratner, "Emerging Issues in Learning Communities," Yellow Wood Associates, Inc. (1997);
- Denis Ralph, "Learning Communities: The Return of Camelot?" Presentation to the Australian National Training Authority National Conference (2000). Accessed at http://www.premcab.sa.gov.au/lifelong-learning/.