Food Standards Australia New Zealand

2   Methodology

2.1     Modifications to the methodology
2.2     Questionnaire Design
2.3     Pilot
2.4     The survey sample
2.5     Response rates
2.6     Distribution procedure controls
2.7     Data coding
2.8     Data processing
2.9     Data analysis
2.10   Confidentiality
2.11   Reporting

The questionnaire was sent to clients of the medical / hospital allergy clinics, private allergy clinics, and members of the allergy support groups by the recruiters who agreed to participate in this survey. The recipient of the questionnaire pack was requested to give the questionnaire to the main grocery buyer in their household to complete. An option was also provided for those under 18 years of age to complete the questionnaire provided consent was obtained either from their parents or guardians.

A comprehensive description of the methodology is included as Appendix A of this report. This section includes information on modifications to the planned methodology, response rates and data management by the researchers.

2.1   Modifications to the methodology

The methodology adopted for the survey design was largely based on that specified in the Methodology Report (Appendix A). However, there were some variations and these are noted below.

  • For the Australian questionnaire, potential respondents were provided with the national free call phone number should they require assistance in completing the questionnaire. They were also provided with the contact details for the FSANZ Project Officer for any further information on the survey, and for the Department of Health and Ageing Ethics Committee if they wished to make a complaint regarding the survey.

  • For the New Zealand questionnaire, potential respondents were provided with the national free call phone number should they require assistance in completing the questionnaire. They were provided with contact details for FSANZ in Wellington for any further information on the survey, the Health and Disability Advocate in North and South Islands, Auckland District Health Board Maori Health Support for assistance or complaints regarding the survey.

  • There was some difficulty in obtaining clinicians / hospitals in Western Australia. As a result, the only responses from Western Australia were derived from contacts within allergy support groups.

  • Before the research team could engage clinicians/hospitals to take part, ethics committee approvals were required from each individual hospital, even though the Australian Government Department of Health and Ageing Ethics Committee initially granted approval for the survey. Some ethics committees requested changes to the survey design that would have impacted on the parameters for the survey for that site. This would have affected the consistency of the bi-national survey parameters. After explaining the reason why the researchers preferred not to change the parameters for individual sites, most ethics committees granted approval, however a few did not. These sites were, therefore, omitted from the survey.

  •  As a substitute for the questionnaires that could not be distributed via hospitals when the time came, an additional distribution of questionnaires, over and above that originally planned, was sought from several support groups and some private clinicians (in both Australia and New Zealand). In New Zealand, attempts were made to include respondents from both North and South Islands and approval was obtained for both sites. But South Island was not officially represented, as the medical site could not identify food allergy clients from its records. An alternative centre where there was an immunology clinic was located, but no immunologist was available to select clients. It also proved not possible to identify any supplementary private practitioners from outside Auckland.

  • A less than anticipated number of private clinicians agreed to participate in the survey when invited and some were unable to participate due to time constraints. Thus, private clinician questionnaires were distributed by fewer practitioners than was initially planned.

  • A set of recruitment criteria were developed for use by medical specialists to ensure that both children and adults with different types of food allergies were contacted from all states, territories and regions with access to allergy/immunology clinics.

2.2   Questionnaire Design

The survey questionnaire was designed to be self-administered with both 'closed' and 'open-ended' questions. Primarily, NFO Donovan Research developed the survey questionnaire with the assistance of two specialist dietitian consultants. The draft questionnaire was reviewed by FSANZ and its Project Team for appropriateness prior to the pilot survey. The Project Team included members from allergy support groups, immunologists, food industry representatives, dietitians, nutritionists and other Australian and New Zealand government representatives.

Two separate questionnaires were developed; one for Australia and the other for New Zealand (Appendices B and C). The questions in both questionnaires were the same except for two key differences:

  • demographics section (section F) was tailored specifically for each country; and

  • question 18 in the knowledge and information section had an additional option of ‘manufactured food database’ (MFD) in the New Zealand questionnaire.

The questionnaire attempted to elicit responses on specific allergens rather than the categories as stated in Standard 1.2.3 (clause 4) of the Code. The allergens included in the questionnaire do not, therefore, correspond exactly with those listed in the standard, for example, only wheat was included in the questionnaire not the category ‘cereals containing gluten and their products’, peanut and soy were listed separately in the questionnaire, as were tree nuts and sesame.

According to the Code (Standard 2.2.3 Fish and Fish Products) ‘fish’ is interpreted as any of the cold-blooded aquatic vertebrates and aquatic invertebrates including shellfish (FSANZ 2002 1 ). In the questionnaire, fish and shellfish were investigated separately.

2.3   Pilot

The pre-test of the survey was conducted at the offices of NFO Donovan Research in Perth (Australia) amongst one focus group of seven potential respondents referred to in this section as participants. The participants were recruited by word of mouth networking amongst the friends of staff and snowballing to their friends. All potential participants were asked to volunteer themselves for inclusion if they were interested in taking part. Thus, all participants were aware of the voluntary nature of the research.

Each participant was posted a package containing the materials that would be sent in the final mail out to 'at risk' consumers (i.e. questionnaire, information letter, and two copies of the consent form). They were requested to complete the questionnaire and then bring it along to the discussion. Participants were paid $40 for their time and input.

Overall, there were no major issues identified with the questionnaire with most participants indicating that the questionnaire was comprehensive and easy to complete. Full details of the pilot survey can be found in Appendix A. There were a few minor changes made to the questionnaire and the results of the pilot survey have not been included in the results of this report.

2.4   The survey sample

For the survey proper, respondents were sourced from Australia and New Zealand in relative proportion of their respective populations. The New Zealand sample was boosted slightly in an effort to attain a total of 100 completed questionnaires.

The questionnaires were distributed in both countries from 11th March 2003, the final cut-off for the receipt of questionnaires being 4th July 2003. The final date had been extended several times during the process to accommodate the delays that were encountered.

The overall sample projections by country are shown in Table 2.4a together with the proportions achieved from each.

Table 2.4a   Sample achievement by country

.

actual POPULATION proportions

total required1
(a )

totalachieved
(b )

.

%

%

%

Australia

83

79

81

New Zealand

17

21

19

.

100

100

100

1         New Zealand sample 'boosted' to achieve 100 completed responses.

 

The sample projections are shown for each state and territory of Australia and for island of New Zealand in Table 2.4b together with the number achieved from each.

Table 2.4b   Sample achievement, by state/ territory or island as proportions of the country sample

.

proportions required
(a )

proportions
achieved
(b )

Australia

%

%

New South Wales

34

30

Victoria

25

22

Queensland

19

21

South Australia

b 8

11

Western Australia

10

a 5

Tasmania

2

4

Northern Territory*

1

0

Australian Capital Territory

b 2

a 6

Total Australia

101**

99**

New Zealand

..

North Island

b 76

a 95

South Island

b 24

a 4

Total New Zealand

100

99**

*         Northern Territory was excluded from survey due to inability to locate appropriate sample.
ab        Indicates categories where there is a significant difference between the results in each column (significance at   the 95% confidence level )
**       Does not add to 100% due to rounding

Due to difficulties in achieving full support from some organisations and individuals, the sample that was eventually attained varied from that planned. However, the overall division by country was similar to that required.  

2.5   Response rates

Response rates for the different sample segments are shown in Table 2.5.

Table 2.5   Response rates by sample segment

.

TOTAL

Australia
(a)

New Zealand
(b)

Base: all respondents

513*

416*

97

.

Mailed out

Returned

%

Returned

%

Returned

%

Source

n

n

%

n

%

n

%

Hospitals

471

180

38

157

42

23

23

Private clinicians

356

115

32

83

30

32

39

Extra private clinicians

87

42

48

42

48

NA

NA

Support groups

172

118

69

83

63

35

88

Free call number**

80

58

73

51

86

7

33

OVERALL

1166

513

44

416

45

97

40

*         3 questionnaires were logged in but later removed due to incompleteness
**       A Free call phone number was set up to facilitate calls to NFO Donovan Research from either Australia or New Zealand.

For this survey, a total of 1166 questionnaires were distributed in Australia and New Zealand and 513 [2] were returned, a response rate of 44% overall. Given the nature of the methodology, this is a relatively sound response rate [3] . A brief review of published Australian literature indicated that when enclosed self-administered questionnaires together with information letters were distributed to potential respondents, response rates ranged from 12 – 67% (Hallidayet al2003; Iverset al2003 & Sernaet al2003).  

As could be expected, the highest response rates were attained by those who called the Helpline (self-selectors) and those who were recruited via support groups. Hospitals and private clinicians attained quite similar responses with around a third returning their questionnaires. Around half of the extra questionnaires sent out by the clinicians (in response to the failure of several hospitals to do so) were returned.

In this survey, it was noted that individual response rates varied considerably between recruiters [4] , raising a further question about whether all the questionnaires were actually distributed [5] by all recruiters. One recruiter who had initially agreed to participate did not distribute any questionnaires, although a number of phone calls were made to confirm participation. This brought the overall response rate down. The questionnaires for this particular recruiter were returned to the researchers unused in November 2003. The questionnaires for another hospital that declined to participate, and notified researchers of this, were able to be redistributed to hospitals in other states prior to the final closing date.

Apart from the issues outlined above, the non-response rate could also have been affected by additional factors such as:

  • the questionnaire distribution relied of the recruiters posting out the questionnaire packs to potential respondents. Some packs may have been sent to addresses from where the potential respondent had moved and not informed the recruiter. However, the number of packs that were marked 'return to sender' was low suggesting that this had a very minimal effect overall;

  • questionnaires might have been sent to households where English was not spoken at home, and some potential respondents may not have completed the survey. Help with the questionnaire was offered to all participants through the NFO Donovan Research helpline numbers in both Australia and New Zealand, and more specifically to Maori speakers in New Zealand;

  • it was possible that some respondents received questionnaire packs from two sources; i.e. medical and support group. In this instance, respondents were requested to complete one questionnaire only;

  • it was possible that some recruiters may not have mailed out the reminder cards one week after the questionnaires were posted, as instructed; and

  • some questionnaires may have been received close to / or past the cut-off date indicated on the cover, thus leading potential respondents to believe that the completed questionnaire would no longer be accepted.

2.6   Distribution procedure controls

As per the methodology report (Appendix A), approval had been sought and received from individual medical institutions prior to the mail-out of questionnaire packs.

The dispatch of questionnaire packs was tightly controlled. Once the total number of recruiters was identified and their individual sample amount was allocated, each recruiter was assigned a series of numbered questionnaires for distribution and these ranges were recorded. This measure was undertaken to maintain some record of where questionnaires were returned from (to check on relative representativeness), and to provide the ability to calculate response rates for each of the sample recruitment points. Recruiters' packages were then couriered or Express posted to the street address provided by each.

As completed questionnaires were received by NFO Donovan Research, individual questionnaire numbers were recorded to enable some limited cross-checking and to facilitate the calculation of response rates. This checking also enabled the identification of recruiters from whom no questionnaires were received, so that follow-up calls could be initiated.

This numbering system identified the recruiter only. The personal or contact details for individual participants were masked for all but the support group sample, whose participants were recruited via the NFO Donovan Research helpline. The contact details recorded by the helpline staff were at no time linked to the results on returned questionnaires. All identification details for this small sub-group, were deleted as soon as the questionnaires were sent out to them.

All except five returned questionnaires were input: two questionnaires were excluded because, through a printing or questionnaire assemblage problem, respondents had not been presented with a complete questionnaire; and three questionnaires were excluded because they had only been partially completed.

2.7   Data coding

The coding of open-ended questions was undertaken by NFO Donovan Research, and some consultation occurred with the contracted nutritionists in the team to ensure the codes used and analysis were appropriate. All coding was checked by a senior coder.

2.8  Data processing

SurveyCraft was used to produce data tables with full significance testing [6] . Data tables were checked by the researcher and any errors or data anomalies detected were corrected.

Market Research industry standard practice was employed for analysis of these research data. The analysis technique used was uni-variate only, i.e. cross-tabulations. No multi-variate analysis was undertaken.

In the methodology report, it was suggested that weighting could be applied to adequately represent the two countries and the states / territories (Appendix A).   Because of concern that there may be differences between respondents from the different sample sources (see 2.2.1) that weighting could distort, it was deemed to be inappropriate to weight in this instance due to the anomalous representation from the three sample sources in each location. The data presented in this report are therefore unweighted.

2.9   Data analysis

The results of respondents' awareness of products or ingredients that contain substances of concern are reported only against the key substance of concern in that product (sections 5 and 7).

This is because products or ingredients may have varied composition depending on the manufacturers’ specifications. For example, marzipan is reported against tree nut allergies only, although it could also contain eggs and wheat; 'surimi' is reported against fish but it could also contain eggs, starch and shellfish. Vegetable oil is reported against peanuts, but not included in the tree nut section. It should be noted thatStandard 1.2.4of the Code states that if the source of vegetable oils are potential allergens, then the source must be declared on the food label (FSANZ 2002 1 );

As part of the analysis a constructed measure was also developed. This measure calculated the number of items a person had identified in two questions (Q14 and Q30) as being an issue for their household. Each possible allergen was counted and an individual's score was compared to the total possible. This was done to give an indication of how well people identified the many substances of concern. Limitations with this are:

  • there was a great deal of 'confounding' evident due to the number of households with multiple allergies;

  • the substances that had been identified as being of concern generally, may not be of concern to an individual (e.g. vegetable oil could include peanut oil but this may only be a problem if it is unrefined); and

  • respondents may not know the meaning of some of the terms (e.g. they may avoid 'crab sticks' but not know that 'surimi' is the name for this substance).

These limitations should be kept in mind when considering the results reported in section 7.7.

2.10   Confidentiality

In accordance with the Privacy Act 1988 (Australia) and the Privacy Act 1993 (New Zealand), the survey methodology that was adopted was designed to ensure the privacy of all respondents. Therefore, the letter contained in the questionnaire pack to participants included statements to the effect that:

  • their participation was voluntary;

  • no personal details were passed on from the medical agency to the research company (medical recruits only);

  • any information collected for the survey would be used for research purposes only;

  • all responses would be anonymous; and  

  • there were clearly stated mechanisms for complaints.

The completed questionnaires were stored in a secure location within NFO Donovan Research (Perth) premises and destroyed as soon as the report had been finalised.

All data are secured on the NFO Donovan Research computer network by restricted access to the relevant directories and password protection. Only individuals who were eligible to access the files i.e. those who administered the survey, coded responses, processed data or wrote reports, were granted access to these data.

During the survey period, several respondents requested a copy of the published report and therefore, provided their names and addresses to FSANZ. These personal details were not linked to the survey data, and their details will be destroyed by FSANZ once the published results have been distributed to them.

2.11   Reporting

The data are presented by sub-group where consistent differences are seen, usually by country, but also by other sub-groups, most commonly:

  • the presence or absence of membership of a support group;

  • the age of the person with the most serious allergy; and

  • the total allergies in the household.

Areas of significant difference are indicated on the tables using superscripted alphabetic column designations, e.g. significance was measured at the 95% confidence level.

The results of this survey are presented under the following headings:

1.          who the survey covers;

2.          identification and diagnosis of the most serious allergy;

3.          knowledge and information about the allergy;

4.          managing the allergy;

5.          the use of food labels to manage the allergy; and

6.          other labelling issues.

Following the results section, limitations of the survey are discussed and the key outcomes of the survey are presented in the conclusions and recommendations section.

Exec Summary  | Section 1 |  3  |  4  |  5  |  6  |  7  |  8  |  9  |  10  |  11  |  Appendixes  |  Full Report pdf 1609 kb