 |
 |
|
|
Proposed Methodology
While this preliminary data may seem to indicate that the process is in some ways not effective in leading to the enforcement of environmental law, and although it does lead to some recommendations for its improvement, all the possible conclusions that the data indicate are by nature highly contextual, and as a result, intriguingly inconclusive.
As you recall our goal in compiling this
preliminary data was to use it to evaluate our pilot methodology in order
to create a methodology for answering our research questions.
Now, using our preliminary data as a basis, the findings that we will
be presenting to our client, and which we will highlight for you now,
are recommendations for how best to assess the effectiveness of the CSP
and how best to determine which changes, if any, would increase this effectiveness.
If we return to our hypotheses for our first question, we can look at these steps individually. Our research focused on two of these steps: public awareness and levels of enforcement.
First, looking at public awareness, our preliminary data were limited in that a large portion of them stemmed from a methodology based on the assumption that media coverage is a good indicator of public awareness. We relied on media coverage as an indicator in order to assess how much public awareness of the issues raised in the submissions resulted from the existence of the submission process, the filing of a submission, and the publication of a Factual Record.
We found that public awareness cannot be gauged based on media coverage for two reasons.
The first is because the relationship between media coverage and public awareness is extremely complex. And second, a media search is extremely difficult to conduct accurately due to the subjective choices required for search terms and databases and because of the subjectivity required to define relevance of an article.
Second, looking at enforcement levels, interviews with submitters and media searches yielded limited findings on the effectiveness of the process in leading to enforcement. Researching enforcement activity should therefore be expanded in two ways:
(1) By researching enforcement efforts of environmental laws by the three governments through interviews with government officials.
(2) By using a baseline study of enforcement levels before the inception of the CSP. This would be a more accurate way of attempting to assess any causal link between environmental law enforcement levels in the three countries and the existence of the CSP.
Enforcement should be gauged in part by indicators such as: the number and severity of fines and the size of the budget for enforcement.
However, we would recommend a careful interpretation of any conclusions drawn from this particular study. This is because data indicating low enforcement levels may indicate a number of things. For example, it may also mean that there is already good compliance with environmental laws. It may indicate that there are low environmental standards. Or, it may indicate that there are simply very few environmental laws to enforce.
The issue of enforcement then leads to a further issue regarding the effect that the factual record may have on environmental law enforcement. Our preliminary data seem to indicate that governments of the three countries make an effort not to cooperate with the process. This seems to suggests that the factual record may be a powerful tool.
If we return to our hypothesis for our second research question, our research focused on limitations in leading to the effectiveness of the citizen submission process.
Our questionnaire was primarily concerned
with the user-friendliness of the process. Because most respondents indicated
that this user-friendliness was not an issue and, in fact, expressed other
concerns regarding the process in line with the literature, we have re-designed
our questionnaire.
Questions referring to timeliness, transparency and toothlessness were
added to those regarding user-friendliness. Also, the questionnaire emphasizes
any recommendations that the submitters may have to make the process more
effective for environmental law enforcement.
The questionnaire also includes questions on the role of the Secretariat,
the council and the government in leading to the effectiveness of the
citizen submission process.
In addition, by only questioning past submitters
about the user-friendliness of the process, we were necessarily considering
a biased sample of potential submitters. There may in fact exist individuals
or ENGOs who considered filing a submission, but chose not to because
they perceived the process as too onerous. The existence of non-submitters
would help indicate the effectiveness of the citizen submission process
in leading to environmental law enforcement.