Assessing the Impact of International Associations
- / -
Printed in Transnational Associations, 1978, 10, pp 435-440 [PDF version]. From a paper, originally presented under the title Networks of International Associations, occupational categories and world problems, which was completely re-written in the light of the papers and discussion at the Conference on International Scientific and Professional Associations (ISPAs) and the International System (Philadelphia, November 1976).
A second part of the paper on the impact of associations, together with material from the original version, has been incorporated into: International organisation networks, a complementary perspective. In: P. Taylor and A.J.R. Groom (Eds.). International Organisation, a conceptual approach (London, Frances Pinter, 1978, pp. 381-413). The quantitative data originally presented has been updated and appears as: International organizations, an overview. In: Yearbook of International Organizations. (Brussels, Union of International Associations, 1978).
This paper considers various aspects of the significance and nature of the impact of international associations on their environment, and particularly the impact of international scientific and professional associations on the international system. In approaching this matter, it is first useful to examine why the question of impact is important, what is meant by impact, and the questions raised by the process of proving impact. This establishes an appropriate context within which to comment on the progressive increase in the number of international associations and their interrelationships and the manner in which networks of organizations may diffuse impact and act as vehicles for its transference.
A. Assessment of impact of INGOs on IGOs
Justification for assessing impact
The following points indicate the major reasons for assessing impact :
1. Policy concern : In order to justify an existing policy with regard to an international association, it is appropriate to assess the impact of the body on its environment. Of a slightly different nature is the need for an organization to assess the general impact of such a body on its environment before responding to an unprecedented attempt by such an association to influence the organization's policy.
2. Resource allocation : To the extent that the allocation of resources in support. of project proposal of a particular association is a program rather than a policy decision, then it may be important to evaluate the actual or potential impact of the association on its environment.
3. Acknowledgment of recommendations : Many associations produce recommendations, resolutions or declarations which may be directly or indirectly transmitted to parts of the intergovernmental system. In order for IGOs to justify attention to such recommendations, they must prove that the association has adequate political impact to give credibility to such positions, irrespective of their content.
4. Suspension of relationships : Under certain circumstances (e.g. ECOSOC's positions in relation to Spain, South Africa and Taiwan and channelling of CIA funds through INGOs), an IGO may need to prove inappropriate impact in order to justify suspension of relationships with an INGO, or some other form of sanction or censorship.
5, Provocation : Since there is a range of INGOs associated with the ideology of each major power bloc, the IGOs associated with a power bloc may wish to prove the negative impact of the equivalent INGOs on any other power blocs as a justification for some form of tacit or overt support. (Where the impact is shown to be positive, this then becomes justification for some form of sanction or censorship as under the previous point.)
6. Value elaboration: Where national or international assocations have built up a climate of opinion superior in some values to those with which the intergovernmental system is associated, IGOs may wish to recover lost ground by proving the positive impact of selected INGOs in order to justify binding them into IGO programmes (the UN approach to the environment issue is a case in point).
7. Reinforcement& constituency: Where IGO member states have for political reasons generated resolutions initiating programmes which alienate much of its usual constituency, it may seek to prove the impact of INGOs on such programmes in order that by so associating them it may establish a favourable climate of opinion for the programmes amongst the INGOs constituencies.
8. Tradition, prestige and public relations: Where an IGO wishes to maintain relations with a particular INGO for special reasons, it may prove impact to justify such a position (the relation between the UN and the World Federation of United Nations Associations is a case in point).
Varieties of impact
The different types of impact can be grouped as follows:
1. Physical, including violent demonstrations, occupation of offices, physical damage to buildings or equipment, violence or threats of violence to personnel, physical assistance (manpower), etc.
2. Affective, including non-violent demonstrations, emotional propaganda, smear or hate-campaigns, supportive campaigns, etc.
3. Procedural, including strikes, lockouts, restraining orders, procedural and regulatory devices (legal, administrative, financial, safety, health), resolutions, declarations, etc.
4. Programme content, namely conceptual or information inputs contributing to the elaboration of programme content, within its predetermined framework.
5. Organization policy, namely political, financial, statistical, conceptual and similar inputs affecting the formulation, selection and rejection of programmes.
6. Policy coordination, namely political and other considerations affecting the coordination of programmes of semiautonomous organizations acting on interrelated problem areas.
7. Research, namely conceptual and methodological advances which effectively question the utility and significance of the problems addressed by existing programmes and policies.
8. Socio-political, namely political, ideological and philosophical advances which effectively question the utility and significance of :
(b) the research by which the problems and remedial action are defined.
The utility of the conventional approaches may also be questioned in the light of comments such as that of Peter Drucker:
It could be argued that many international associations function in order to change meaning, to support or facilitate any such change, or to maintain continuity through such changes. Their success in doing so is not necessarily detectable by the methods of evaluation normally recommended. Moreover alternative philosophies may well change the significance, if any, of "success" as determined in this way and the legitimacy of actions based on conclusions of "low impact".
Related to the indirect forms of impact noted above is the static impact which in its most extreme form is now termed structural violence.
Issues raised in assessing impact
The process of proving and assessing impact raises a number of issues which are briefly reviewed here :
1. The situations in which a demand is made for an assessment of impact tend to be structured such that impact must effectively be proven before attention is directed towards the bodies giving rise to the impact. The "existence" of such bodies is deduced from the recognition of the impacts to which they give rise. If no impact can be detected then the question of whether such bodies exist is considered irrelevant. The convenience of this approach does not eliminate the question of whether the organizational system has an adequate concept of its environment, in that some impacts may be undetectable by the methods or criteria used, and some external unrecognized bodies may suddenly give rise to impacts for which the organization is unprepared.
2. Related to the previous point is the assumption of absence of impact on an organizational system unless impact can be proven. It is certainly debateable whether this is an appropriate attitude for an organization (as noted above) or for the intellectual disciplines associated with the assessment and its methodology. It is particularly unfortunate in that the assumption places the burden of proving impact on the external unrecognized body (in a manner somewhat analogous to that of a legal system in which innocence, rather than guilt, has to be proven).
3. The demand for proof and assessment of impact places the body making such demands in a special position in relation to those who may be perceived as having impact. Where such bodies have a special place in the international system (e.g. the United Nations), the conclusions of any such evaluation effectively contribute to the definition of the reality of the international system. Those bodies excluded from this reality by this process have no method of appeal, since the effects of the evaluation process are not of interest to the bodies demanding it. Such evaluations may usefully be termed "directive assessments" because of the by-products of the evaluation process. It is important to render explicit for whom a particular set of impacts is considered significant and in whose interest.
4. Impact studies are organized in terms of impact on a focal organization or group (known as the point of anchorage in social network analysis where it is usually taken to be some specified individual whose behaviour the observer wishes to interpret). This raises the question of what bodies are undetected or ignored by this approach, whether such bodies may have some indirect impact on the focal organization, and whether the behaviour of the whole set of bodies in a network does not effectively result in diffusion of all impacts throughout the network.
5. Current impact studies necessarily predefine what processes are to be considered as conveying valid impacts. This raises the question of what other processes are undetected or ignored by this approach and the consequences of inability to focus on them.
6. Impact studies raise the question of how the thresholds are selected below which impacts of a particular kind are considered insignificant. (The physical sciences are fortunate in having established how "weak" and "strong interactions" should be taken into consideration, thus enabling them to give appropriate attention, for example, to the impact on an object (a) of a failing weight, (b) of a mass any specified distance from it, and (c) of weak electromagnetic forces such as the magnetic field of the earth. The question may be asked whether impact studies in the social sciences are able to focus on impacts analogous to (b) and (c) where there is no direct impact as such merely the influence of forces, which under some conditions, in the case of physics, may be of considerable significance aside from being necessary to any adequate understanding.)
Studies of association impact on the intergovernmental system raise the question as to how relevant the impact of one organization on another is to an understanding of their separate or combined impact on the problems for which they were established. The approach loses sight of the fact that society's available institutions are failing to contain the complex of problems on which they purport to focus.
It is difficult to avoid the general impression of a series of continuing sterile debates about "pseudo-issues", effectively (although not deliberately) structured to avoid converging on conclusions which could legitimate any recommendations for remedial projects to increase the value of organizations and associations separately and as linked in networks. Such issues can be termed --- pseudo-issues, because, from a very realistic and practical point of view, there is little that can be done about any of them individually at this point in time. Such issues should better be seen as constraints on any action strategy, rather than the prime policy concern in connection with INGOs as tends to be the case in IGO, INGO and academic circles. Hopefully many of these problems will be overcome at some stage, but it would seem to be unnecessarily shortsighted to allow them to constitute delays to effective development of the full potential of the INGO network. The organizational instruments for action may in many cases be imperfect, but concentrating attention on their imperfections may simply obscure the fact that they are already quite adequate for many tasks and that the specific imperfections are in large part a circumstance of the times rather than of their nature. Practical approaches to improving their ability to perform their functions may well be the quickest method of reducing their imperfections. The point made here has been explored elsewhere (7).
Conventionai evidence for impact
As noted earlier, there are problems in obtaining satisfactory evidence of the impact of international associations on the intergovernmental system, particularly since within the IGO system such evaluations tend to be tied to programme themes such as development, environment, peace, human rights, etc.
The category of scientific and professional associations is not used by the IGO system, although occasional references are made to technical assocations. It is interesting that probably some white coilar trade unions coming within the parview of ILO could also be considered as professional associations.
There have been numerous positive statements concerning international associations in general, produced by officials from the UN Secretary-General downwards on appropriate occasions, as well as from government delegates. Official resolutions frequently call upon such bodies for some action or support. Unfortunately none of this constitutes "evidence" of impact, because such statements may always be interpreted as having a public relations component. Although if this is the case, the obligations felt by parts of the IGO system to maintain good relations with such associations may perhaps itself be considered as stronger evidence of impact. Assessments by scholars do not in general, for reasons noted earlier, provide good evidence for the presence or absence of impact, except in the case of intensive study of particular associations or groups of associations (cf. the studies of Edward Miles of space, telecommunications and sea-related bodies). IGO secretariat assessments, such as those of ECOSOC and UNESCO, of NG0s in consultative status are basically descriptive rather than evaluative.
Thus, although it would be possible to select, sift and cite specific statements of positive impact, the question remains as to whether this would be considered positive proof (and by whom) or merely circumstantial evidence of little relevance to current theory in the field of political science or policy studies. Current theories are indeed indifferent to such evidence. For example, Keohane and Nye note that the impact of inter- societal interactions and transnational actors in international affairs has been ignored in both policyoriented writings and more theoretical works, and that when they have been recognized they have often been consigned to the environment of inter-state politics, and relatively little attention has been paid to them in their own right or to their connections with the inter-state system (8). Singer and Wallace are quite explicit about exclusion of NGOs from their analysis: "our interests (and, we suspect, those of most of our colleagues) are more concerned with IGOs than with nongovernmental organizations" (9).
Finally there is the question of what criteria to use in evaluating the evidence for possible impact of ISPAs on IGOs. Should the criteria relate purely to the transfer of scientific knowledge and considerations ? Should they relate to science policy and use of resources for science ? Or should they simply relate to political clout irrespective of the scientific and professional component ? Curtis Roosevelt, former Chief of the NGO Section of the UN Secretariat, makes the point (10) that IGOs are political institutions and an NGO can only be effective in relation to them by relating to such bodies politically. The reality of the situation is that governmental delegates assess the potential value of an NGO primarily in terms of the political power of the constituency it represents. Scientific or professional expertise does not necessarily imply political power. Furthermore, most expertise, however technical, is now held by IG0s to have political overtones. Even NG0s concerned with astronomy, cardiology or sanskrit literature, for example, are not effective in IGO terms unless they take positions on issues such as peace, human rights, etc. Clearly an NGO or ISPA low on expertise might therefore be perceived as having more impact than one having high expertise and little political sensitivity. What would be a good indication of political impact in this context ? For example, the ability to influence the wording of a resolution is an indicator of impact, but what if the resolution is never effectively acted upon by the IGO (as can be frequently argued). The abilily to influence allocations of funds is, also important' but what if the resources are small relative to the expenses of the lobbying activity necessary (as is the case with many programmes of interest to ISPAs) ?
The disadvantages of following this route seem clear enough, and in the light of the argument of the previous sections another approach seems more appropriate.
Characteristics of impact-oriented associations
It is perhaps useful to distinguish a category of international associations whose operations are strongly influenced by the desire to impact directly upon the intergovernmental system. Such associations tend to have characteristics such as the following:
A category of international associations whose operations are not strongly influenced by the desire to impact directly upon the intergovernmental system may also be distinguished. Such associations tend to have characteristics such as the following:
Some studies of the impact of international associations on the intergovernmental systems employ a procedure which results in misleading, if not erroneous, conclusions. An impact study may be organized in terms of one of the following, for example :
Assumption I: that because part of the intergovernmental system has given rise to an organization, a programme or a conference to focus on a particular subject or problem, then any international association which attempts to act on that issue would want to interact with the structure in question. This is incorrect because a significant number of international associations may consider that the particular structure (i) can itself only be relevant to a (possibly minor) aspect of the issue, (ii) has been prepared, or operates, in such a way that most decisions of any significance are either taken in advance or in other arenas, (iii) is conceived mainly as an exercise in public relations to focus public support and the attention of some governments insensitive to the issue, (iv) is conceived as a political compromise substituting for any effective action on the issue.
Assumption II: that because an international association is represented at some intergovernmental organization, programme or conference, then the association is necessarily attempting to have an impact on that intergovernmental structure.
This is incorrect because a significant number of international associations may consider that the structure suffers from the defects identified under the previous point. In order to maintain a line of contact with the intergovernmental body, whilst minimizing the resources engaged, they may effectively employ any of the following strategies :
(ii) allow the association to be represented whenever necessary or convenient by whatever member happens to be living in the area or passing through;
(iii) allow the association to be represented by any enthusiastic member interested in the activity for personal reasons (including personal status and prestige, etc.);
(iv) allow the association to be represented by a non- member with some special interest (e.g. conducting interviews for a research project).
Assumption III: that because a representative emphasizes the interest of his association in having impact on some intergovernmental organization, programme or conference, that the association necessarily has such an interest or that any of its efforts at impact are related to the representative in question.
This is incorrect because:
(ii) the representative may feel obliged to disguise the minimal response of his association, if he is aware of it, to avoid negative consequences for his association;
(iii) the association may feel obliged to be represented to ensure that it is still recognized as "in the game" by its peers, by any part of the intergovernmental system which makes later use of the mailing lists, and possibly even some of its own members or by those conducting studies of representation which may be widely distributed;
(iv) the association may participate not in an attempt to have impact on that intergovernmental body but in order to counteract any impression of bias arising from its special interest in interacting with some other part of the intergovernmental system (e.g. with a different ideological orientation).
This is incorrect because:
(ii) the association may not have a position on the matter as well-formed as is implied by the ability to respond to questions about it in particular
(iii) the association may not conform to a conventional structure and be easily comparable with its peers;the structure may be so loose that it is not possible for an individual to speak on behalf of the association as a whole.
This is incorrect because an intergovernmental secretariat has a number of offices (in the case of the larger agencies) or positions via which it interacts with associations. The lack of coordination between such offices is well recognized. Such offices may include:
(ii) a bureau responsible for consultative relations with NGOs, in support of
(iii) a governmental committee defining which bodies shall be called NGOs, and defining policy on them, which may be ignored by
(iv) departments concerned with substantive programme areas working with useful associations, irrespective of whether they are "NGO" or international,
(v) departments emanating, receiving or exchanging information with associations
(vi) the agency conference environment in which a particular assocation may get considerable air-time through several government delegates.
Thus when an intergovernmental representative complains that the associations with which he has any contact (possibly at agency-convoked meetings) are naive, he may well be correct. Agencies have set up such an unfruitful environment for contact with associations that the latter avoid contact because there are more effective forms of action. Those that do not either have special introductions to exploit (and are therefore assessed as "effective") or are in the process of learning what a waste of effort such contacts can prove to be.
It is unfortunate that the process by which the social and policy sciences accord attention to organizations (or problems) in society appears to be so strongly governed by the information handling capacity of those for whom the conclusions are hopefully intended, rather than by any desire to explore the numerous existing organizations and interactions in all their rich variety. This question has been explored elsewhere in connection with the perception of world problems (11). In attempting to articulate their dissatisfaction with current studies of international organizations in 1968, Keohane and Nye "felt that an 'Everest syndrome' prevailed. Scholars studied organizations simply because 'they are there'. We agreed that new approaches were needed."
Their book is testimony to their success (8). The remark remains valid however. Big impacts on big organizations are studied because they are so visibly there. The reluctance to consider less visible phenomena is strengthened and supported by a posture requiring unequivocal proof that the phenomena are there before any such inquiry can be entertained. It is an interesting question as to how much national and international NGO activity is required before it becomes theoretically interesting or of significance to policy formulation, and how much an adequate response to problems is delayed by such conceptual lags on the part of those who should be ensuring the necessary conceptual leads to anticipate emerging structural changes.
1. Stafford Beer. The Cybernetic Cytoblast: management itself. September 1969 (Chairman's Address to the International Cybernetics Congress).
2. Scott Boorman. The Protracted Game; a wei chiichi'lapproach to Mao's revolutionary strategy. Oxford University Press, 1971.
3. Scott Boorman. Outline and bibliography of approaches to the formal study of social networks. Harvard University, 1973 (Eels Discussion Paper 87). pp. 239-287.
4. Peter Drucker. The Age of Discontinuity; guidelines to our changing society. Pan, p. 11.
5. AIvin Toffler. Value impact forecaster a profession of the future. In: Kurt Baier and N Rescher (Eds). Values and the Future. Free Press, pp. 1-30.
6. Johan Galtung. Feudal systems, structural violence and the structural theory of revolutions. Proceedings of the IPRA Third Conference. Assen, van Gorcum, 1971.
7. Anthony Judge. Practical problems in using the potential of INGO networks. In: The Future of Transnational Associations from the standpoint of the New World Order. [text]
8. J. S. Nye and R. O. Keohane. Transnational relations and world politics. In: J S Nye Jr and R 0 Keohane (Eds). Transnational Relations and World Politics Harvard University Press, 1972.
9. M D Wallace and J D Singer. Intergovernmental organization in the global system. 1815-1964; a quantitative description. International Organization, 24, 2, Spring 1970,
10. Curtis Roosevelt. The political future of transnational associations; the opportunity for effective NGO action. In: The Open Society of the Future: report of a seminar to reflect on the network of international associations. Brussels, Union of International Associations, 1973, pp. 91-96 (Originally presented to a Conference of Nongovernmental Organizations in Consultative Status with ECOSOC, Geneva, 1972). Brussels, Union of International Associations, 1977.
11. Yearbook of World Problems and Human Potential. Brussels, Union of International Associations,1976, (See Section P). [commentary]
this work is licenced under a creative commons licence.