Non Award Bearing Continuing Education

Interim Evaluation Report

This is the first report of the evaluation of the Non Award-Bearing Continuing Education programme of the Higher Education Funding Council for England, prepared for HEFCE by NIACE.

The purpose of the report is to examine the bidding process itself, with a view to informing HEFCE future practice. It does not examine the wider issues about the programme, its purposes and impact, which will be pursued in the main report.

Background

Adult Learning in Higher Education

Until the abolition of the binary divide in 1992 adult learners were funded quite differently in the Universities and the Polytechnics. In the Universities (with the exception of small numbers of mature students on mainstream degree courses), adult learning was generally the province of specific Departments of adult education or continuing education, with a range of functions, and a history of separate direct funding from Government. In the Polytechnics, on the other hand, adults had always been well represented in most courses, and their education was assumed to be part of the function of all Departments, with no distinct funding mechanisms.

There was some overlap in terms of the kinds of provision offered by the two sectors, but relatively little exchange of expertise and knowledge, and some mutual suspicion. Separate funding mechanisms and separate national representative organisations (UCACE and PACE) tended to limit cross fertilisation between the sectors.

Thus, in 1992 the new funding Councils inherited two distinct funding methodologies and an expectation of equitable treatment for all, without any evident agreement about how this might be achieved. To address the problem HEFCE and HEFCW issued a joint consultative paper Continuing Education: a Policy Review in 1992. In the light of the responses HEFCE announced its intention to encourage the "old" Universities to "mainstream" their course provision, incorporating them wherever possible into the normal systems of quality assurance and accreditation of the Universities. This would enable them to be funded as part-time courses through the normal HEFCE formulae for teaching provision. It did, however, have the effect of creating a rather sharper distinction between "degree level" and "sub degree" programmes which could not be accredited by HE institutions, and were thus to be unfunded (through HEFCE) in future.

Institutions were given no guidance about what proportion of their work might appropriately be treated in this way, but in the event the large majority of the work was mainstreamed (in some institutions 100%). The funds related to this provision were transferred to the mainstream grant to the relevant institutions, and the balance became one of the sources of funding for the present programme. No evidence was sought as to their understanding of the implications of this, nor of the way in which institutional quality assurance processes were to be applied.

The Non Award Bearing Continuing Education Programme

Following the mainstreaming of most continuing education work, HEFCE agreed to allocate the remaining balance of £1.6M to "non award-bearing provision", and to supplement this with a further £3M for "access related provision" and "work targeted at disadvantaged groups". This created a fund of £4.6M available for non-award bearing CE. By comparison with institutional continuing education budgets, this was a relatively large sum to commit to developmental work, although some would argue that it is a small response to the major under representation of some groups in HE.

The Council issued Circular 4/95 Funds for Non Award Bearing Continuing Education in February 1995, inviting institutions to bid for work under either or both of the themes, now designated "Liberal Adult Education" (LAE) and "Widened Provision" (WP - the title subsequently changed to "Widening Provision").

99 institutions submitted 154 bids (90 in WP and 54 in LAE), proposals were assessed by an advisory group convened by HEFCE, and 65 were approved for funding (45 in WP and 20 in LAE). HEFCE subsequently invited tenders to undertake the evaluation of the whole programme, and the contract was awarded to NIACE. This is the first report of that evaluation.

This fund was only one of several initiatives of HEFCE and its predecessors to encourage wider participation in HE, and develop continuing education at HE level. The existence of these other programmes may affect what institutions expected from the Non Award-Bearing CE programme, and may partly explain why few bids were submitted on some topics. Relevant initiatives include:

HEFCE also inherited the former DfEE PICKUP programme and has allocated £14.9M in 1995-6 to continuing vocational education. In addition, the early 1990s saw a rapid expansion of Access to HE courses, encouraged partly by the Government’s commitment to expanding participation, and partly to the rapid development of the national structures for the recognition of such courses. Continuing Education departments played a significant role in the development of such courses, although most are, now mounted by FE, rather than HE, institutions, and funded through the FE Funding Councils.

Evaluation Methodology

This evaluation study is being carried out in two phases. The first, of which this is the report, is concerned only with the bidding processes. The second is concerned with broader issues of impact, value for money and effectiveness.

Phase 1 was carried out in two parts:

Responses were obtained in one or other form from 83 bidders, representing a 54% overall response rate.

The project team comprises six members of NIACE’s permanent research staff (Stephen McNair, Veronica McGivney, Alastair Thomson, Sue Cara, Jim Soulsby and Alan Tuckett). The first five all conducted interviews for this stage of the project.

The team agreed a preliminary set of questions for the semi-structured interviews. The aim was to generate qualitative, rather than quantitative evidence, and the standard questions were modified for different groups of interviewees.

The initial interview sample was selected to secure a balance of LAE and WP, and a range of successful and unsuccessful bidders, aiming for at least 5 interviews for each category of bidders, with a bias towards successful bidders. As far as possible they were also spread across a range of different kinds of project and different regions. Where the same individual had been responsible for bids under both WP and LAE, they were asked about both. This accounts for some imbalance in the final sample, but there is no evidence to indicate any consequent distortion in the final results. Overall 23% of all bidders were interviewed.

At HEFCE the two senior policy staff responsible for the programme were interviewed, as were the three administrative staff who had been involved in the process.

Three members of the Advisory Group appointed by HEFCE to assess bids were interviewed by telephone. Altogether the following groups were interviewed or responded to questionnaires:

* excluding HEFCE staff (who include the Chair).

The postal questionnaire comprised a condensed version of the standard questions. It was sent to the person identified as the contact for every bid received by HEFCE, including those which were to be interviewed (some returned questionnaires and were also interviewed).

Findings

This section of the report outlines the findings which emerge from the various interviews with bidders, HEFCE staff and group members.

Purpose

HEFCE sought to achieve a number of purposes, most of which were understood by bidders.

The motivation of HEFCE in creating the programme was assumed in the Circular, and not made explicit. HEFCE interviewees commented on it that "left to its own devices the system would not respond to HEFCE policy objectives", and that "the Council was anxious to preserve valuable non-accredited work".

There was also an underlying need to be seen to be dealing fairly with institutions from both sides of the old binary divide, which, in the case of LAE, included giving those previously excluded a fair chance to bid for funds, and those who had not mainstreamed all their provision a chance to make up their shortfall.

These tacit agendas were generally understood by most bidders, some of whom suggested that the Council had been surprised by the extent of mainstreaming, and felt that perhaps it had gone too far, risking the loss of important work, and that this programme was an opportunity to retrieve some ground.

There was a considerable amount of direct and indirect consultation before bids were submitted. Those active in national circles were familiar with the extensive discussions about mainstreaming and brought their own interpretation of that debate to bear, while others consulted HEFCE staff before bidding. In the event, the criteria were interpreted in a range of ways, and one HEFCE officer felt that about 1 in 6 of bidders misinterpreted the purpose significantly.

Liberal Adult Education was the most contentious and ambiguous area.

In respect of Liberal Adult Education the circular described the purpose simply as:

to support courses which are not mainstreamed and where students’ attendance is primarily for interest in the subject and for personal and social development, rather than vocational objectives

It gave further guidance that:

those institutions which have recently mainstreamed 100% of their work previously funded from separate CE funds will need to provide a strong rationale for additional funds

In the case of liberal adult education (LAE) bidders views were mixed. Some felt that the funds were to "mop up" work which had not been mainstreamed, while others saw it as an opportunity to reinvent liberal adult education in more creative ways. There was not a clear expectation that all the work funded would be developmental nor that institutions would be changing their approaches as a result of it.

LAE was undoubtedly the area of greatest political sensitivity. The complex history of liberal adult education and of mainstreaming, and cross binary tensions made the LAE programme particularly prone to interpretation, rumour and misinformation. Although the bidders in general thought that the Circular had been clear, and several described it as self explanatory, they interpreted it in very different ways, reflecting their experience of recent history and their reading of the politics of the unified HE sector. Some assumed that priority would be given to those old Universities which had mainstreamed a smaller proportion of their work, and had therefore lost money (and at least one HECE officer took this view), while others took it as an indication that HEFCE wished to encourage the development of LAE provision in those institutions which had previously been ineligible (because they had been Polytechnics). This created a set of conflicting expectations which are reflected in some of the comments made in interviews following the announcement of the results. On either side of the former binary line there were bidders who believed that the programme was designed to favour the other.

Widening Provision was less contentious.

In the case of Widening Provision, the guidance was slightly more extensive:

development activity which will lead to wider participation in higher education for groups previously disadvantaged in this respect. It is expected that a wide range of such provision will receive funding. Such funding will be largely activity based rather than driven by student numbers, although direct provision is not precluded. It may include, for example, student support, both academic and personal; outreach and community work; Compact schemes and Associate Student Schemes.

WP bidders were clearer about the purpose, probably reflecting a broader consensus among providers and policymakers about the underlying agenda. However, there were no bids for several of the developments suggested as examples.

Group members took the view that the widening participation monies were to pursue the strategic aim of increasing access from non-traditional learners. It was felt that this funding was for developmental and innovatory work which should produce significant change in institutional approaches by the end of the programme.

The general view of bidders was that HEFCE was seeking to encourage innovation, rather than merely to maintain or expand existing provision, although telephone interviews did not suggest that bidders generally had given much thought to whether such funding should be directed towards cutting-edge innovation; to endorsing and encouraging the transfer of existing "good practice"; or to encouraging institutions with little or no tradition in liberal adult education or widening participation to experiment in this area.

It is worth noting that a number of bidders did not appear to have considered the underlying intentions of HEFCE or the programme as a whole. These simply saw it as a funding opportunity broadly consonant with their own priorities. In more than one case, bidders said that they had not considered what HEFCE’s broader purpose might be.

Bidding Process

The bidding process was as follows:

The programme was announced in February 1995 in HEFCE Circular 4/95.

Potential bidders were invited to consult HEFCE staff for clarification of the Circular (and many did).

99 institutions submitted 154 bids. 91 under WP (totalling £10.8M) and 64 under WP (totalling £5M). 55 bid under both headings.

45 institutions did not bid at all: including 1 new University and 7 old ones, and 9 of 31 general colleges. The remaining non-bidders were specialist colleges.

HEFCE convened an Advisory Group who assessed bids against the criteria in the Circular, and graded them on a scale from A-D.

Bids graded A were funded for 75% of funding requested, Bs were funded for 53% of funding requested, and bids graded C or D were not funded.

Institutions adopted a variety of top down and bottom up approaches to bidding

There is no consistent pattern in how bids were prepared in institutions, and internal processes varied widely depending upon organisational structures, cultures and traditions. Approaches included both "top down" and "bottom up". In the former, the decision to bid was taken centrally at a senior level and the drafting carried out either by a central development/ strategic planning unit or by particular departmental staff assigned the task. A number of institutions have some form of central planning office which maintains a central register of initiatives seeking funding and matches them against all incoming invitations to tender. In several cases the initiative matched long standing plans by the institution, reflecting the concern of particular senior staff. In these cases much previous groundwork had already been done, and in some cases bid drafting was carried out by Pro-Vice Chancellors. One HEFCE officer commented that there were "a lot of senior signatures".

In other cases the HEFCE circular had passed through various offices before the decision to bid was made by an individual lecturer or head of department who then drafted a bid before going on to secure the support of their departmental peers and the endorsement of senior institutional managers (Academic Registrar, Pro-Vice Chancellor or Deputy Vice Chancellor). For some of these the bidding process provided an opportunity to raise the profile of the work within the institution: one questionnaire respondent wrote that he was "probably most anxious about getting the bid approved by the University in the first place because of its limited familiarity with such work and the fact that (the topic) was not perhaps a top priority".

Competitive bidding was seen as the "least worst" way of allocating funds of this sort

Most respondents accepted that, given the scale of funds available, competitive bidding is the "least worst" way of allocating funds. Few commented that they disapproved of the principle, and none suggested any alternative mechanism. Although CVCP might wish to argue against this kind of "top slicing", there was a strong view that the impact of funding in this way is very beneficial, and represents value for money in relation to HEFCE’s objectives.

In the main, the process was viewed as helpful. Most interviewees believed that it had helped focus thinking within the institution, encouraging them to clarify priorities and objectives with regard to different target groups; to explore the costs of implementing particular priorities and to encourage debate and reflection about the direction and rate of change within the institution. Although the Circular did not require bids to be explicitly innovative, a number of bidders commented that the bidding process had prompted them to think hard about development and innovation. A number commented that the work would never have been funded by the institution without this initiative, and a small but significant number of unsuccessful bidders said that some of the proposed work had gone ahead, in full or in reduced form, despite the lack of HEFCE funding, as a result of the planning carried out to prepare the bid. Some bidders, and some HEFCE staff, felt that the sums involved would not have had the same impact disbursed through normal funding channels, and that this justified the procedure when dealing with such limited funds.

There was less confidence about this among LAE bidders, especially among unsuccessful ones, reflecting the view among those interviewees that the purpose of LAE funding should have been to compensate for funds lost through mainstreaming. Those who merely bid for "more of the same" were disappointed.

Group members had a pragmatic view of the use of competitive bidding for the distribution of these funds. They considered that processes involving field work and visits to institutions would be excessively time-consuming and uneconomic. They were conscious that some institutions were very experienced in the writing of bids and this might mean that those would obtain more significant sums. However, they felt that this was appreciated by all involved and that they could only adjudicate on the bids as presented to them.

It was felt that the four year project length was reasonable and did allow institutions to demonstrate their ability to perform. Some interviewees commented that the bidding process might not have been cost effective for a shorter period of funding.

Concerns expressed were more to do with the way the mechanism was used than the principle itself. Several interviewees commented on the time available for preparing bids, especially since bidders were encouraged to seek partnerships with other agencies, which might call for time consuming negotiation and consultation. This gave an advantage to those institutions with previously developed plans and relationships on which they could call rapidly. If this were the case it would, perhaps properly, favour those institutions seeking to use the resources strategically over those whose bids were purely opportunistic.

As might be expected, those whose bids were successful were more likely than others to view the preparation of bids as being time well spent. In general the opportunity to generate new ideas and approaches with colleagues was seen as valuable by respondents. One commented that , "discussion of key issues and principles took place within a department not universally comfortable with such work". However, more than one respondent noted that the scaling down of funding had led their institution to question whether the effort put into bidding to HEFCE might have been more productively been directed elsewhere.

The advantages of competitive bidding identified by bidders included:

The disadvantages identified included:

Further points recorded included:

The bidding process is expensive, but justifiable

Competitive bidding is an expensive business. HEFCE staff have to prepare a draft Circular, consult, revise and issue it. Institutional staff have to prepare ideas, draft and consult on bids. HEFCE staff and advisors have to read, assess and review bids, checking for consistency of scoring and fairness of process. Decisions have to be recorded in case of subsequent disputes. Where proposals are not fully funded, staff time is involved in renegotiating outcomes and work plans.

All sample institutions were asked to estimate the amount of time involved in preparing bids, and a question on this was included in the questionnaire. The data must be interpreted with care (interviews were conducted long after the event, often several people were involved in the process) but Widening Provision bids appear to have involved substantially more time (mean of all respondents 9.5 days) than Liberal Adult Education (mean of all respondents 4 days). The range extended from 1 to 50 days.

However, the question is not a simple one, several institutions pointed out that writing bids could be done quickly only because of earlier groundwork on the preparation of strategy and objectives, and the nurturing of links with external partners. Thus the bid itself was only the end of a long process of building partnerships and speculative planning in anticipation of finding a funder. One University suggested that such preparatory work had probably involved the equivalent of 3 months full time work, before the Circular was received . It is likely that the bidding process favoured such institutions, but this may well be appropriate: those who have been developing plans and partnerships for some time probably have greater commitment, and may well be more likely to achieve results, than those whose interest is aroused for the first time by the Circular.

Group members provided their time on a voluntary basis, and gave significant amounts of time on this process. Some read all the bids and all spent days on their reading. If reading time and time spent at meetings (including planning meetings) were added each member spent between 9 - 12 days on the process. However this time was commonly considered worthwhile and developmental. Group members welcomed the opportunity to extend their own knowledge of the field and current development thinking, and to contribute to national development.

Assessment of Bids

An advisory group was necessary and effective, though there are sensitive issues

HEFCE convened an advisory group to read and assess bids. The group was chaired by the Head of Policy at HEFCE, and comprised individuals whose professional roles gave them an overview of the HE system as a whole, or who were familiar with the field but detached because they worked in Scotland or Wales.

Group members had some concerns about the membership of the group (these concerns were not raised by bidders or HEFCE staff, who commented that the group had been unusually conscientious). The degree of concern depended to an extent on their particular perspective. There was a feeling that it was unfortunate that institutions with projects involved in the bidding were on the group, a feeling that there were too many members from outside England and a feeling that the group would have been enriched by an informed member from outside the education sector or from research and development within HE.

In a specialised field of this kind it is clearly wise of the Council to convene an advisory group to assist in selecting successful bids. One HEFCE officer commented "we are not equipped to argue about bids, we are civil servants". Another described it as "introducing a necessary process of peer review". However, there are a number of problems with the approach. Firstly, group members give their time voluntarily, and there are limits to how much demand the Council can make on them. The process therefore needs to be streamlined, and the volume of documentation kept to a minimum. This is inevitably in tension with the need to ensure that bidders are able to present a proper proposal and adequate evidence of their credibility. Although this is a proper concern, HEFCE officers commented positively on the diligence of group members, and one commented that it was "one of our better groups".

Secondly, in a relatively small and incestuous world like adult education, most of those who have appropriate expertise already know some of the institutions, and often the bidders, personally. This raises issues both about impartiality and about extraneous, but relevant, information. This is particularly problematic in the world of adult higher education, which has traditionally been sharply divided across the binary line, and individual group members are likely to have much more extraneous knowledge about one or other sector. One suggestion to deal with this was that membership should be extended to include people with expertise in research in HE, or from other sectors of education like FE or Adult Education.

The Council sought to address the issue of impartiality by asking group members to declare interests, and special connections with institutions, in advance of receiving bids, and not to participate in discussion about those bids in the selection meeting. Every bid was read by two people, neither of whom had direct interest in the relevant institution, and their scores compared. Each bid was read by two group members, distributed to ensure that each group member shared at least one bid with every other member. There was a considerable difference between "hard" and "soft" scorers on the group which only became evident at the final group, and a simple standardisation process was applied to adjust scores. However, this was very much a last minute adjustment, and clearly would have been better done before the meeting itself. Some panel members felt that this should have been done, and moderated scores circulated in advance of the meeting to enable members to review the implications. In the event this did cause some unfortunate disruption to the group meeting. HEFCE recognise this problem and have put procedures in place to avoid it recurring.

The issue of extraneous information is more problematic. Members of the Advisory Group were aware that they could not help but bring outside knowledge to their deliberations but took a clear decision to try to base their recommendations only upon the evidence presented even though, in the words of one member, this meant they were aware of "certain places underselling themselves". This approach is problematic, since to keep strictly to the documentation favours the "professional bid writers" perhaps at the expense of those with more established expertise in the work, but to allow such extraneous information into the discussion probably favours those institutions which are well known and highly regarded at the expense of those new to the field. Although no one interviewed doubted the good faith and professionalism of the group members, some concern was raised about whether sufficient information was available to make good decisions. One suggestion was for a two stage process, whereby a longlist of bids would be produced on the basis of outline proposals, and only those institutions asked to produce full proposals, which could be scrutinised in more detail. Some bidders, however, argued that this would duplicate effort, and seek to apply a degree of rigour to a process which everyone recognises will always be imperfect.

Some HEFCE officers, and some group members, expressed some concern about the group processes themselves. Time was necessarily short, and there was a view that insufficient time was spent discussing criteria before reading bids, and that when it met, the group spent too much time discussing bids which had been given low scores by both readers, leaving inadequate time for decisions about borderline cases. This was exacerbated by the last minute discovery of systematic variations in group members scoring.

Some group members read all the bids but others concentrated their efforts on their allocated number. All those involved were impressed by the thoroughness of the members of the group in reading and evaluating bids and within the limitations of time and the process itself group members felt that the process had been fair and consistent. The briefings given by HEFC Officers were seen as very adequate and , indeed the servicing of the group HEFC was seen as excellent.

The group were surprised at the size of the bids and in general at the quality, particularly at those in the sphere of widening participation. However they were also conscious of a number of sketchy and ill prepared bids and a certain proportion of bids were considered to be opportunistic, from institutions which had not evidently thought through their policies or proposals. There were also cases of LAE bids for "more of the same", which were not well received by the panel.

The selection criteria were generally seen as appropriate. They were also the criteria applied by the group.

The Council’s Circular 4/95 set out the criteria used for assessing proposals.

a. Evidence that the proposed provision meets the needs of the target group, community or area. For liberal adult education bids, the special nature of the proposed activity and how it meets needs otherwise not provided for. b. Evidence that provision of high quality will be made (for example, that quality assurance procedures are in place and that there is appropriate infrastructure support). c. Evidence of institutional experience of widening provision and liberal adult education. For liberal adult education bids the nature of the relationship between proposed activity and mainstream provision will be considered. d. Evidence of collaboration with other bodies where appropriate. e. Evidence that provision is consistent with, and related to, an institution's mission and objectives and that proposed activity will be embedded within the institution. f. The bid price.

The Circular also asked for further information:

13. Bids for both liberal adult education and widened provision should also provide information on the following:a. The relationship of proposed activity with the mission and objectives of the institution.
b. Evidence of partnership and collaborative arrangements with other bodies such as FE colleges, Training and Enterprise Councils (TECs), the Workers' Educational Association (WEA) and Community groups.c. Activity in both liberal adult education and widened provision will relate to particular target groups, communities and geographical areas. Institutions should clearly specify the focus of the proposed activity and provide evidence of how this will meet their needs.d. How the quality of provision will be assured.e. The planned timetable for the introduction and development of activity over the funding period.

However, it was not clear what use was to be made of this latter set of information, and the principal criteria used by the Group members were the former ones, with (as the Circular indicated) a weighting towards the first three. The same six criteria were used both for both Widening Provision and Liberal Adult Education bids. Group members scored each bid against the six criteria, which were weighted (to give higher priority to the first three) they were then further weighted to reflect systematic differences in scoring between group members, to produce an aggregate score for each bid.

Interviewees from institutions, questionnaire respondents and group members all accepted these criteria as essentially appropriate and fair, and no suggestions were made that they were applied inconsistently, although some institutional respondents believed that, in the case of LAE, older Universities seeking support for provision which had not been "mainstreamed" would be preferred to new Universities.

In interpreting the criteria there is evidence to suggest that the group members were in practice, applying criteria which were not explicit in the Circular. Two of these were not mentioned in the Circular - degree of innovation, and clarity of outcomes and methodology. The third was the extent and nature of partnerships, which drew on the evidence presented under paragraph 13b. However, as one HEFCE officer suggested, the peer group existed to interpret the criteria in the light of their own knowledge of the field, and respondents did not suggest that these were inappropriate ways of interpreting the overall intention of the programme..

There were some problems in interpreting the criteria

One Assessor commented that, overall, WP bids were of a noticeably higher quality than those for LAE.

Although bidders were asked to indicate their track record, most were not clear about the purpose of this, being uncertain about whether HEFCE was seeking leading edge development in institutions with an established record, or development in institutions entering a particular field for the first time. Some unsuccessful bidders felt that insufficient attention had been given to their previous record. For some "old" Universities which believed themselves to have a strong record in these fields, and which had not mainstreamed all their existing provision, a failure to achieve LAE funding on this basis was particularly bruising.

There was some doubt about what kind of evidence was appropriate to support claims of partnership and collaboration, although both group members and HEFCE wanted to encourage it. Some proposals included formal letters of support from potential partners while others simply listed organisations with which they had worked in the past. One group member felt that some degree of steer might have been desirable with regard to the sort of partnerships that would be welcomed and felt that, in the absence of such guidance the evidence that was supplied was "not a deciding factor" when bids came to be assessed. In interviews it was apparent that some institutions which had spent considerable time liaising with potential partners felt that the resulting evidence had not been given due weight by the group. On the other hand, the evidence of the successful proposals suggests that such evidence was used.

A number of proposals went to some lengths to demonstrate the innovative nature of the proposal even though this was not requested - thus suggesting either that some bidders had misread the brief or that they assumed additional criteria would be applied beyond those set out. In the event such efforts seem to have paid off.

The understanding which members of the Advisory Group had of their function may have affected how they assessed bids. One member commented that the purpose of the exercise was to "pursue a strategic objective of making institutions more accessible - thus weighted to systemic change not pilot projects" - a distinction that may not have been clear from the circular, and not matched by the HEFCE officer who commented that the programme was "project oriented rather than system oriented". Another member also suggested that one of the purposes of the exercise was to "mop up that small proportion of (LAE) work that had not been mainstreamed - an insurance policy" something which supports some institutions’ beliefs that newcomers seeking to enter the LAE field could have been at a disadvantage.

Group members were satisfied that the outcome of the process had been fair and rigorous, but there was some unease among them about the process, which was felt to be too rushed. There was felt to be a lack of clarity about the status of LAE bids from institutions which had mainstreamed 100% of their work, and inadequate prior consultation among group members about the interpretation of the criteria before they scored bids. The failure to identify significant differences in scoring between members until the day of the meeting led to a confused final group meeting, and a sense that the priority to reach a conclusion on the day had perhaps forced them to make rather less sophisticated decisions than might have been the case.

Outcomes

There was much innovation, especially under WP

The purpose of competitive bidding is to identify the best quality bids, rather than to distribute resources evenly across institutions or the country, or indeed across themes or approaches to development. Nevertheless, the nature of distribution is worth noting.

Successful Liberal Adult Education bids have a number of factors in common. With the exception of bids from specialised institutions there is considerable evidence of partnerships with other statutory providers and in a number of cases with the voluntary sector. In the case of bids in categories A and B this is particularly pronounced. Only one bid in these categories outside the specialist institutions did not make such collaboration overt. Many institutions had a long history of partnership in the particular areas of work that were being bid for.

There was a large range in bid price among the successful bidders but the vast majority of bids were at about £1500 per FTE or below. A significant number were below £1000 per FTE and only one above £2000. There did not appear to be much difference between successful and unsuccessful bids in terms of costs.

Innovation was not a requirement of Liberal Adult Education bids, but this was nevertheless a feature, especially in relation to new and specialist institutions. There were a number of developments in provision of art and design which were new and obviously demonstrated an extension of the curriculum on offer to learners. There were also some instances of provision aimed at particular client groups which had not been tackled before and in some bids collaborative strategies were indicated which shared innovative approaches.

There were however, even among the successful bids in the Liberal Adult Education programme, a number of projects which seemed to be the remains of a mixed liberal adult education programme, left after accreditation, refocused and repackaged. In this regard projects targeting older learners and rural areas were perhaps the most obvious examples. The older Universities with their long tradition of extra-mural work demonstrated this most although other older Universities had very innovative projects.

Within the widening participation bids there was a great deal of innovatory activity and evidence of new ways of working in the strategies proposed in the successful bids and this differentiated them from those which were less successful. Even where the basic proposal was not new (like, for example, the creation of an access and guidance unit) the successful bids addressed or developed such proposals in new and different ways. Some of the bids were for projects which had obviously been developing over time and were looking for a funding possibility but this resulted in carefully though out proposals with commitment from institutions and did not, in the case of successful bids result in the funding of irrelevant or non-developmental work.

A number of projects made use of existing students as a resource in supporting others and as role models for possible future students. There were a number of interesting projects involving distance learning and the new technologies which were specifically targeted at non-traditional learners outside the University’s own premises. Many of the guidance and access proposals were noticeable for their acceptance and understanding of the need to set up networks with other educational providers and community groups. There was also an exciting group of projects dealing with ways to enable access to maths, science and technology for non-traditional learners. Other curriculum areas, e.g. languages, art and design did not receive specific funding although there were proposals for them particularly from the specialist colleges.

Success was spread evenly across new and old institutions, but colleges were less successful

HEFCE staff commented that a number of institutions not previously prominent in work of this kind had bid, and some been successful (which was seen as a positive outcome). They also commented on the evident enthusiasm of bidders for the work, and the diversity of issues and client groups addressed.

Almost all Universities bid under one theme or both (1 new University and 8 old Universities did not do so). In WP there was little difference in overall success between old and new Universities, but in LAE the old Universities were notably more successful. Colleges were less likely to bid (11 out of 31 general colleges did not bid), and they were notably less successful. Specialist colleges, where they did bid, were even less successful (see table below). Interestingly, in view of the strong views on "hidden agendas" expressed by some bidders, HEFCE took the view that this outcome would "help counter the view that HEFCE is caning the old Universities".

About one third of bids were primarily concerned with central, rather than departmental initiatives.

There were significant regional differences

The group did not seek to ensure any particular geographical pattern in the allocation of funds. The eventual outcome does not reflect any deliberate decisions, but is nevertheless striking. Some regions were conspicuously more successful than others:

the North East was most successful, with all institutions bidding, and all funded for at least one project (9 bids successful among 5 institutions)

the North West was least successful in both LAE and WP (at 11% and 31%)

more than half the bids were successful in East Midlands, South East and South West

fewer than one third were successful in London, North West and South

only in East Midlands and South West were LAE bids more successful than WP

success in WP was more concentrated regionally, with half the successful bids in three regions (London, SE and NE).

Widening provision bids were stronger and more successful

Widening Provision bids were more successful than Liberal Adult Education ones: about half of the former and one third of the latter were funded. This result is reflected in interviews with group members who commented that WP bids were, in general, of a higher standard than LAE ones. Successful LAE bids were concentrated in old Universities (over half of all successful bids - 12), although there were 20 bids from new Universities and 15 from colleges and specialist institutions, and 9 of these did succeed.

Although the funds for the NABCE programme came from two distinct sources - the balance of £1.6M remaining after funds for mainstreamed provision had been transferred to institutions’ main grant, and the further £3M identified by the Council for access and disadvantaged work - there was no explicit intention to distribute the resources on this basis (a 35:65 split). However, although some bidders assumed that the funds allocated to the two strands would correspond to these two sums.

In the event, the balance in numbers of funded bids matched this (31:69), but the sums allocated were biased against LAE (17:83), reflecting the group’s view of the relative quality of bids, and the larger sums bid for under WP. The average sum allocated per bid under LAE was £40,250 (range £7-99K), while under WP it was £84,350 (range £10-189K).

The successful bids covered a broad range of themes

There was no attempt by HEFCE or the group to prioritise particular themes, and indeed some group members felt that it was a particular strength that institutions could determine appropriate local priorities which they should address. The bids ranged widely. Not all mentioned specific target groups or approaches but those which did so related to students from across the age range from school students with low aspirations to adults from disadvantaged groups and ethnic minorities. The most common themes (in order of frequency of mention in proposals) are:

Ethnic minorities 19 references
Guidance 13
Rural 11
Technology in learning 8
Third Age 6
Compacts/areas of low achievement 6
Art, design, performing arts and media 6
Science, maths and engineering 5
Women 4
Disability 4
Special needs 4

Successful Bids

The bids which were successful were more likely to:

There was some disappointment among Group members that there were very few bids that demonstrated collaboration between two or more institutions within the sector and it was felt that joint bidding might be encouraged in any future funding round.

The reactions in the postal survey do bear out the suggestion that there are those who are well practised at bidding and there are those who are not, although it is not clear that this had a significant bearing on the outcomes.

Unsuccessful bids

There was an expectation among a number of group members and HEFCE officers, that some of the proposed work would be undertaken in those institutions who were unsuccessful, and a small but significant number of unsuccessful bidders said that they had managed to find alternative ways of carrying out some of the work, through redistribution of priorities or alternative funders. However, this was not true of a majority of those interviewed. Many of the unsuccessful bidders did express a desire to hear more about the successful projects, and to learn from their experience.

Funding

Bid price was not a critical factor, and the group chose to spread benefits thinly across more projects

Bidders were asked to provide bid price information as follows:

LAE

Bid price per student. The bid should relate directly to student numbers and the bid price per student will be a factor in the assessment of liberal adult education bids. The bid price should be shown in relation to full time equivalent students (FTEs) and should include overhead costs (para 8c circular 4/95)

Widened Provision

Bid price. The bid price should relate to the costs of the proposed activities. Overhead costs should be included. It is expected that most bids will range between £50,000 and £250,000 and institutions seeking lower or higher amounts should explain why their bids are exceptional. (para 11d Circular 4/95)

Part of the criteria (para 14f) was bid price, although (para 15) stated that

The Advisory Group will pay special attention to the first three criteria.

The Advisory Group agreed that, in view of the strength of bids, the best value for money would be achieved by part funding a larger number of bids, rather than concentrating funding on a smaller number at full price. As a result, category A bids were reduced by 25%, and category B bids by 47%. One HEFCE officer suggested that the Circular should have been more specific about its guide prices for bids, to make it easier to make realistic comparisons between them.

Some group members were happy that less funding was offered to bidders than they had applied for and assumed that this would be dealt with by the reduction of outcomes or the attraction of additional funds. Other members were less happy and particularly with the differentiation between differing grades of bid which was seen as arbitrary in some cases. There was a general expectation among the group that some of the unsuccessful projects would be carried out anyway.

There is evidence of overbidding, but this is not universal

The Circular did not indicate that a reduction would be applied to bids, although most bidders were not surprised by it. About two thirds of respondents said they had expected it, and a significant number said that they had deliberately overbid in anticipation of this. This was not, however, universal.

Although all the project contract prices were within the guide price range specified in the Circular, it is clear that some successful bids were originally outside the range. It is not clear whether the Group considered this at its meeting.

Other comments from the telephone sample included:

"built it in"
"historically we always get less so we compensated"
"one usually over bids"
"over bid by 25%"
"not aware of deliberate over bidding"
"delighted to get what we did"
"the process encourages game playing"

That so many expected a reduction must be worrying because as the reactions to the next question will indicate many actually levered up their bids to take that into consideration. The trick as one interviewee said "guesswork is required as to the likely level of reductions". It is therefore possible that those more experienced at dealing with HEFCE and also perhaps involved in national networks may be more skilled at making that sort of guess, although there is some evidence that such informal networking misled some bidders.

Most, but not all, bidders scaled down their planned outputs and staff

Although one HEFCE officer saw the scaling down as a test of institutional commitment, most handled the reduction in funding by reducing their costs, usually by cutting planned staffing levels. In a few cases, projects made up the shortfall by some other means. Only one institution declined the reduced funding. In one or two cases key staff were delighted to receive any funding. One interviewee, in an institution which had prepared its proposal in a very "bottom up" way, commented:

We had to reduce the scale of the work and extend the time scale. Mind you, the multicultural coordinator was so delighted that we got some funds that he would still have tried to do the lot on even less than we asked for. Perhaps if he’d been twenty years younger he would have succeeded!

Another commented

We .persuaded the University to reduce its overheads….reworked the wording so that it said ‘we aim’ rather than ‘we will’.

About two thirds of projects coped with the reduction by scaling down their work or reducing intended outcomes (usually by reducing staffing levels). About one sixth sought other funding and a further sixth absorbed the costs within the institution.

Despite the numbers who admitted to over bidding, only one admitted that the reduction made no difference. There was no negotiation of the reduction, although bidders were required to indicate whether they accepted the reduction, and how the project was to be altered to accommodate it. One institution refused the reduced sum.

As a result of pressures of work at HEFCE, information on the reductions was not reported back to the group, who thus do not now know precisely what projects have committed themselves to.

Feedback

Bidders would have liked more feedback on results

The formal feedback to bidders was provided in August 1995 in HEFCE Circular 16/95 Non Award-Bearing Continuing Education Funding: the Outcome. This provided the bare details of institutions and funds allocated. No details were given of project titles or outlines. Some bidders has, at the time of interview, no more information than this, and there was a general view that more information would have been welcome. This was particularly the case among unsuccessful bidders, some of whom believed that previous record should count substantially in funding decisions, and could not recognise a correlation between successful bidders and such known records.

HEFCE staff did produce summaries of successful bids, and a brief report on the overall pattern which could usefully have been circulated more widely, and would have provided more opportunity for networking. The failure to produce this was a result of pressures of other work at HEFCE, and the Council does intend to make project summaries available through the Council’s Website.

Other respondents had gleaned information from personal networks, and some had learned more from attendance at the FACE Conference in Plymouth in July 1996.

There was no guidance given to group members by HEFCE about the confidentiality of discussions and decisions. Some bidders did receive some informal feedback in this way, and it may be appropriate for HEFCE to consider issuing guidance on future occasions.

Dissemination

All would welcome more dissemination

The process of collecting and assessing bids generates a considerable amount of information and knowledge about the ideas on development which are current in institutions, and by the end of the group meeting, members have a good picture of the current "state of the art". No formal developmental process exists to distil this expertise, or use it as a basis for feedback to bidders. The council might wish to consider doing this on future occasions.

Similarly, there were no plans indicated in the Circular to bring projects together to share ideas and experience over the life of the programme. To some extent this is likely to happen through the normal networking of professional academics, and through bodies like UACE and FACE. Some such dissemination also forms part of the present evaluation project.

General Conclusions

Most bidders did not take a broad view of the underlying policy intentions of the Non Award Bearing Continuing Education (NABCE) initiative, When asked to explain the purpose of the scheme most did not talk unprompted about any broader social or economic objectives of the programme, and chose to concentrate only on their individual institutional purposes. One advisory group member commented on the absence of reference to European or international experience or parallels, and there were almost no bids which involved partnership or collaboration between HE institutions, even though there were cases of similar bids from two Universities in the same city.

Successful bidders were more likely to stress innovation, involve several partners with clearly defined roles, present clearly researched evidence of needs, clear targets and outcomes, and demonstrate both strong past experience and its relevance to the new task. This was clearer in the case of Widening Provision (WP) than of Liberal Adult Education (LAE). Of the LAE bids, those which were most like WP ones in terms of innovation and partnership were most likely to be funded, though some would appear to contain a significant element of "repackaging" traditional liberal adult provision.

However, some group members commented that, although the successful bids were in general the most innovative, innovation was sometimes relative, and that some bids were for innovation which would be regarded as unremarkable practice in FE or AE, or elsewhere in HE. There was concern about the "parochial" perspective of some Universities, which were felt not to be aware of good practice in other parts of the HE sector or outside it, or of international experience.

There was no evident bias towards old or new Universities, in WP, and in LAE the bias towards the old universities probably reflects their historical strengths in this kind of provision. Colleges and specialist institutions were generally less successful. Projects were not evenly distributed geographically, but this would appear to be more a matter of individual institutions’ strengths rather than any regional factors (geographical distribution was not a selection criterion).

In general most of those involved thought that the decision to allocate funding through a competitive bidding process was right in principle (or at least the best solution given the level of resource available overall), and most believed that the results seemed fair as far as can be judged. Although the bidding process was very time consuming there were no alternative proposals for distributing a fund of this size, and there was a strong view among successful bidders that the same volume of funding distributed through mainstream funding would never have found its way to these specific targets, despite their centrality to HEFCE policy and institutional missions. This was rather less true of the unfunded LAE bidders, for reasons discussed below.

The time involved in bidding varied considerably, but was not as contentious an issue as might have been expected. Most bidders accepted it as inevitable, and did not feel that the time involved was disproportionate (although there was no consistent relationship between time estimated for the work of bidding and the size of funds sought or received ). It is perhaps significant that bidders under WP claimed to have spent, on average, twice as long (9.5 days) on preparing their bids as the LAE ones (4 days), and that the Advisory Group identified a significant difference in quality between them.

Although the Advisory Group felt that the outcomes of the selection process were broadly fair, and HEFCE staff felt that the group had done a conscientious job, the role, operation and constitution of the Advisory Group was a cause of concern to some of its members, and by implication to some of the unsuccessful bidders. There was no suggestion of impropriety, and no one suggested an alternative to the use of such a group, but there were anxieties about the difficulty of being entirely impartial, when some bids come from institutions whose record was well known to some members while others are from institutions not actively involved in the established networks of CE. The panel’s decision to make judgements on the basis of the formal proposal alone was fair, but probably meant that some institutions with well known reputations did less well than they might have expected.

Some bidders would have preferred a more thorough selection process, to investigate factors like the feasibility of what was proposed and the institution’s track record. The idea of a two stage bidding process (as used by ESRC, and sometimes DfEE) received a mixed response. Its advantages would be a simpler process, sifting out of weak proposals with minimum effort for bidder and HEFCE, and the possibility of giving more detailed scrutiny to a short list of full proposals. However, this would have significant implications in added work for HEFCE and panel members.

Mainstreaming has caused many tensions within institutions and the HE system as a whole. Some bidders believe it has led to the abandonment of important work, and that the LAE funding was not an adequate instrument to protect essential provision. Some LAE bidders felt that institutions have been able to mainstream provision without an adequate test of their ability to do so. It was argued that those who had recognised the difficulties of mainstreaming, and who consequently bid for a lower proportion of mainstreamed provision, should have been more directly compensated through LAE.

There are staff in some institutions who still feel very strongly about the outcome in relation to LAE. The complex political background, and considerable prior debate and rumour had led some to expect particular results (and many of these were disappointed). Many bidders were fairly cynical about HEFCE’s need to placate particular lobbies (though different people had diametrically opposed views about which those were). Some of those with an established record in this field appeared to assume that it would be recognised and valued, and that funding would be given to make up the shortfall following mainstreaming without a sustained argument in its defence. On the other hand, some of those in new universities saw the programme as an opportunity to enter the field for the first time but without any very clear idea of what it meant. In the event neither was likely to be funded, and both were likely to feel aggrieved.

It is clear that there is no consensus about the meaning of "liberal adult education", and a range of distinct notions is in current use. This is complicated by the overlap between this territory and both continuing vocational education and mainstreamed part-time. The absence of any real clarity about what was intended in the Circular led to confusion, and any intention to preserve some unique existing forms of liberal adult education, at risk as a result of mainstreaming, was probably undermined by the decision to handle the funding of WP and LAE through a single bidding process. The LAE bids which were funded were, in general, those most like WP proposals, stressing innovation, partnership and targeting, and the same result could probably have been achieved through a single fund directed at widening provision . Distinct existing forms of liberal provision would have been better protected if HEFCE had identified a separate, ring fenced, fund, defined more precisely what kinds of provision it was aimed at, and processed bids separately. One clear conclusion is that before funding further work in this area, HEFCE should review its policy, with a view to developing clearer rationale for what is to be funded and on what basis. One aim of the evaluation study is to inform such a review.

Some concern was expressed at the absence of clear strategies for learning from projects or disseminating the outcomes. It was suggested that if HEFCE is to distribute funding in this selective way, the results should be available to inform the sector as a whole. Some initiatives, like the joint FACE/UACE conference in July 1996, and the evaluation project itself should help with this, but there was a view that more dissemination, at an earlier stage, would have been useful. Two advisory group members commented on the absence of a mechanism to use the considerable volume of information gathered by advisory group members in the course of appraising bids.

The process revealed evidence that cross institutional networking is less widespread and effective than one might have hoped. Some respondents and advisory group members were familiar with the work of particular institutions, but that knowledge did not always spread across the old binary divide, and the colleges appeared to be in general outside this community. In some cases it would appear that unsuccessful bidders who felt that their track record was insufficiently valued, were unaware of the state of development in other institutions and sectors.

Recommendations

The following recommendations arise from the findings of the first phase of the work, and reflect discussions among the research team, the first project seminar and the project Steering Committee. Were HEFCE to be considering repeating the exercise it might be appropriate to:

1. further clarify the purposes of the programme, including how far it seeks to:

advance knowledge rather than to establish practice
advance leading edge practice rather than to support those catching up
spread funding geographically, or across themes or client groups

2. clarify the mechanisms, including:

how institutional track record is to be demonstrated
whether bids will be funded at full price or liable to reduction
retain competitive bidding as a mechanism for allocating project funds

review the bidding process itself, to consider whether it can be made more economical and effective, possibly through the use of a two stage process, without increasing the time demands on institutional staff and advisory group members.

draw the membership of the advisory group more widely, retaining the principle of peer review but extending the peer group to include those with experience in related areas of post-school education (staff with, for example, experience of curriculum and staff development or institutional change in HE, of development work with adults in FE or AE, or research into HE).

plan the procedures for selecting bids and agreeing funding levels more clearly in advance, to ensure that, in advance of the final meeting, significant areas of disagreement are identified, panel members’ scoring is standardised, and the principles on which bid prices are to be reduced are understood.

design a dissemination strategy at the same time as designing the programme. This should identify the potential audiences for dissemination, include provision for more rapid and detailed feedback on successful project proposals so that the successful can network quickly and effectively, and the unsuccessful can see what kinds of work are being supported. A strategy should include dissemination and networking between projects, between academic and administrative staff working in relevant areas, and senior institutional managers.

design an evaluation strategy at the same time as designing the programme, with the aim of ensuring that evaluators are in place at least when the first projects begin, so that appropriate initial benchmarks can be established.

provide clear guidance to advisory group members on confidentiality, and on their role (if any) in providing feedback to bidders.

We also recommend that HEFCE should review its policy on funding non-credit bearing education, in the light of the experience of mainstreaming, the NABCE and CVE programmes, to establish the rationale for such funding, the levels at which funding should be provided, and how it should be allocated. The findings of the later stages of the current project will include a commentary on these issues.

Top of page