FY07-09 proposal 199803100

Jump to Reviews and Recommendations

Section 1. Administrative

Contacts

Section 2. Locations

Province / subbasin: Mainstem/Systemwide / Systemwide

Section 3. Focal species

Section 4. Past accomplishments

Section 5. Relationships to other projects

Section 6. Biological objectives

Section 7. Work elements (coming back to this)

Section 8. Budgets

Itemized estimated budget

Total estimated FY 2007-2009 budgets

Cost sharing

Section 9. Project future

Future O&M costs:

Termination date: None
Comments: This is an ongoing coordination proposal to assist CRITFC member tribes in working with the NWPCC and BPA to satisfy tribal trust adn treaty rights obligations.

Final deliverables: Final report will be prepared. Database and maps will be shared with subbasin planners and tribal fish and wildlife partners within the Basin.

Section 10. Narrative and other documents

Reviews and recommendations

ISRP PRELIMINARY REVIEW (Jun 2, 2006)

Recommendation: Response requested

NPCC comments: The proposal is to implement the fish and wildlife programs of the Columbia River Treaty Tribes as coordinated through CRITFC. It provides a clear description of the role of the treaty tribes and the need for coordination among them. Without coordination there might be overlap and discontinuity of activities within and between the Tribal F&W departments. Extensive rationale is provided relating this proposal with the objectives of the Fish and Wildlife Program, BiOp, US v. Oregon, CBFWA, Pacific Salmon Treaty, PCSRF, PNAMP, and NOAA Fisheries Recovery planning. The proposal does a good job of specifically identifying CRITFC’s role in relation to each of these. Interactions between the CRITFC watershed group and PCSRF, PSC, PNAMP and individual projects are clearly described. A project history enumerates a long list of accomplishments related to coordination, watershed assessments, proposal assistance, cost sharing, M&E guidelines, Salmon Corps, and Outreach. Reflecting earlier ISRP comments, there are clearly many good coordination activities being conducted through this project, but the proposal, and especially the project history, lacks evaluative content. How were decisions made as to where to focus efforts? What activities are the most effective? What challenges face these coordination efforts? How are they addressed? The proposal would be strengthened by this type of evaluation of the effectiveness of previous actions. The project has a number of objectives related to implementation of the fish and wildlife program: coordination, outreach and education, technical review, database management, and report writing. Methods described for each objective seem reasonable; however, some are too generally written to understand what is entailed. For example: • "Watershed Department staff participates in the development of a Framework for Performance Measures/Indicators for the PCSRF projects" - What methods were used to develop this framework and is it working? • "So far under the PCSRF for FY 2000-2003, some of the progresses the tribes have made include: 442 stream miles restored, 1977 acres acquired and protected, 34 fish passage barriers removed, and 178 miles of riparian plantings" - Under guidance from CRITFC, are the Tribes using a quantitative method to forecast improvements in fish survival? • "Methods under this proposal will be adapted as the NWPPC, CBFWA, and 13 Tribes processes evolve." - Can this statement be made more specific? M&E is a big missing component of this proposal. Methods for assessing successful coordination as it translates to benefits to fish and wildlife need to be developed and used. Earlier ISRP comments recommended that the project would be improved by taking a more targeted approach to implementation. This would involve developing priorities. It also recommended a plan to monitor project effectiveness. How else will the project determine if it is being effective or if there are areas of possible improvement? The narrative states under Objective 1 "Maintenance of cooperative relationships between the CRITFC tribes and the co-managers of natural resources in the Columbia Basin will be evidenced by, among other things, development and implementation of collaborative proposals, productive interaction between the parties and continued improvement in communication and information exchange." These may be suitable metrics to assess coordination success, but the proposal should present a definitive provision to actually monitor them. The proponents could also gather biological data (e.g., escapements, productivity, abundance) from the various Tribes and synthesize them annually as a contribution toward monitoring the streams under their purview. Effectiveness of a coordinating entity such as CRITFC is sometimes difficult to assess quantitatively. Adaptive management and review of past activities may be one way of moving ahead. As suggested in the last ISRP review, CRITFC should develop evaluation methods or perform a literature review to find out what similar agencies do to assess effectiveness of their coordination activities. Measures of effectiveness might include observations on themes such as reduction in overlap, number of collaborative projects with Tribal fishery biologists resulting in peer reviewed papers, and biological improvements such as increases in smolt-to-adult survival rates (SARs) for salmonids returning to streams managed by the Tribes. Despite the good narrative descriptions contained in the proposal, more specific details are needed. More could be done to present examples of CRITFC coordination by adding information from the attachment “CRITFC Success under the Pacific Coastal Salmon Recovery Fund” to the proposal narrative. In addition, sponsors are requested to provide more detailed information addressing the areas listed below: • Tabulation of the numerous collaborative projects done under the auspices of CRITFC. This table could include a column of the technical reports or data resulting from the work. • Methods used to prioritize coordination activities. • Methods used to evaluate the effectiveness of coordination. • Information on effectiveness monitoring; how the evaluation procedure could be improved. • The relation of funded projects to benefits for fish and wildlife. • Potential for information transfer among CRITFC members to be enhanced by electronic means, such as an electronic newsletter to provide more timely information and coordination.

ISRP FINAL REVIEW (Aug 31, 2006)

Recommendation: Fundable (Qualified)

NPCC comments: CRITFC provided helpful answers to many ISRP comments. The response concerning outreach was well done. The list of over 150 completed or ongoing projects is impressive. However, no lists of technical reports or data resulting from these projects could be provided because of the "limited time frame." It is surprising that CRITFC does not routinely have this information available. Better evaluation and documentation of the effectiveness of previous coordination efforts and project implementation in the form of feedback from the four Tribes and other agencies could help CRITFC to identify those activities that have been most effective and to prioritize future efforts. But overall, the response misses the point and does not address the ISRP’s comments on the need for better self-evaluation and monitoring of CRITFC activities. The statement: "It is impossible to clearly state what the most effective activities are" is disconcerting in a coordination project, and can only true if no attempts to evaluate effectiveness are made. Approval of projects by the CRITFC Commission does not constitute an evaluation. The sponsors need to take a more proactive approach to learn how to conduct an effectiveness evaluation and to conduct it. At present, effectiveness is asserted rather than documented. Responses #12 and 16 address some potential indicators of effectiveness, but these remain assertions rather than demonstrations of effectiveness. If it is the case (response #12) that "Effectiveness may well be measured by the success of preserving the tribal institutional capacity and leadership to deliver on-the-ground projects, collaboration to make shared decisions with state and federal co-managers on key policy issues, participation in forums that shape future actions by BPA and other federal entities that oversee the operation of the hydrosystem, and education and outreach to build and sustain partnerships," the elements of this statement provide guidance as to the types of indicators that would be appropriate to assess performance. Response 17 also addresses the effectiveness evaluation issue. Stating, "As already agreed to by the ISRP, monitoring of coordination effectiveness is difficult to evaluate quantitatively" is again missing the point. Although it is difficult, it is both desirable and possible. The point is that careful thought should be given to what effectiveness would look like and how it can be measured, then develop a plan to measure it and evaluate it. Agreeing to "document any incidences of overlap or redundancy with CRITFC and individual tribal projects if they occur as a measure of effectiveness" is not sufficient and does not address the central question of effectiveness. The response provides no indication of a prioritized approach to planning. Planning is apparently entirely reactive to short-term priorities expressed by CRITFC members. Response 15 describes some of the elements of consideration in coordination but does not explain the process of prioritization. The recommended qualification to funding is that the sponsors be required to develop an effectiveness evaluation plan.