# Methods

This note records how version 1 of populon.org chooses, stores, and compares ethnic categories. The tone of the site is descriptive. The same rule applies here: the notes say what a source publishes and what this file does with it.

## What is not done

There is no world taxonomy. A group is stored under the name and classification system of the publishing office. The interface may place two countries on one map, and it says that the categories are not equivalent.

Published counts are not mixed with model output. Since version 2 each country page also has a projection to 2100. It is populon.org's own cohort-component model, built on UN WPP 2024 rates and census age tables, and labelled on the page as model output rather than an official projection. Its method and assumptions are in `docs/projection.md`. The age pyramids show published census tables, or the UN WPP 2024 estimate where no census table by group is loaded. There is no migration-flow model.

Migration background in Germany and the Netherlands is not relabeled as ethnicity. Scheduled Caste, Scheduled Tribe, religion, and language in India are not relabeled as ethnicity. Nationality of foreign residents in Japan is not relabeled as ethnicity. Those countries appear with an explicit statement and no chart.

## Priority of sources

1. National statistical office census tables, or the office's own release of those tables.
2. A later official communiqué that states a comparison with an earlier census, when the earlier integers are the published comparison rather than a new extraction.
3. A compilation that reproduces an official table, only where this note says the office PDF was not re-opened, and only when the compiled integers sum to the official total and match the office's published percentages for the largest groups.

Wikipedia is not a source in this version.

Each wave stores the office, a title, a URL, the year, a type (`census` or `official_estimate`), and a confidence tag. The confidence order used in the interface is:

1. Official census
2. Official estimate
3. Research estimate
4. Compiled / third-party
5. Unknown

No wave in version 1 uses the last three tags. They are in the schema so a later file can use them without pretending a survey is a census.

## Universe

The population in a country file is the universe of the classification, which is not always the headline population of the country.

| File | Universe |
| --- | --- |
| USA | Resident population of the 50 states and the District of Columbia. Puerto Rico is excluded. |
| CHN | Mainland census population in NBS Communiqué No. 2, including servicemen and excluding Hong Kong, Macao, and Taiwan. |
| GBR | England and Wales usual residents. Scotland and Northern Ireland use different ethnic-group questions and are not added in. |
| CAN | Persons in private households. Visible minority is a 25% sample, weighted and randomly rounded. |
| AUS | Census usual residents. The total is the sum of the Indigenous-status components. |
| SGP | Residents: citizens and permanent residents. Non-residents are outside the ethnic table. |
| MYS | The whole census population. Ethnicity is published for citizens. Non-citizens are a separate row and are not given an ethnic group. |
| IRL | Usually resident and present. People temporarily absent are outside this table. |
| NZL | Census usually resident population count. Level 1 ethnicity allows more than one response. |
| FJI | The ethnic wave is the 2007 census. The 2017 census population is stored as the later population total and is not given an ethnic distribution. |
| ISR | End-2023 official estimate of the population of Israelis, rounded to the nearest thousand as published. |

## Country decisions

**United States.** The rows are the Census Bureau's cross-classification of race and Hispanic or Latino origin, so the rows sum to the total. Hispanic origin is not a race. "Some other race" and "Two or more races" are kept as their own rows. The 2010 and 2020 group ids match, and `comparable_series` is empty, because the Census Bureau has said the race distribution is not directly comparable after changes in the question and in coding. The years are shown side by side.

**China.** Communiqué No. 2 (11 May 2021) gives Han 1,286,311,334 and ethnic minorities 125,467,390. Those are marked preliminary by NBS and sum to 1,411,778,724. The 2010 counts are those figures minus the increases printed in the same communiqué (Han 60,378,693 and minorities 11,675,179). They sum to the 2010 mainland total of 1,339,724,852. That pair is joined. Nine large minority counts from the later detailed tabulation are listed under the 2020 minority aggregate. That tabulation's universe is 1,409,778,724 because servicemen are not classified. The detail rows are not added into the chart. The residual detail row closes the minority aggregate; it is an arithmetic remainder of that list, not a census category called "other".

**Brazil.** The concept is self-declared color or race: branca, preta, parda, amarela, indígena. Parda is a color category. The 2010 counts, including 6,608 undeclared, sum to 190,755,799. For 2022 the file uses the color-or-race counts reported from the 22 December 2023 release: parda 92,083,286, branca 88,252,121, preta 20,656,458, amarela 850,130, and indígena 1,227,642. The 2022 total in the file is the sum of those five numbers, 203,069,637. IBGE also published an expanded indigenous count of 1,694,836, which includes people who gave another color and then said they consider themselves indigenous. That count is not used, because it does not partition the population. The 2022 total should be rechecked against SIDRA table 9605. The waves are joined because IBGE compares them, and the page says the 2022 questionnaire added follow-up questions for amarela and for indigenous identification.

**United Kingdom.** The file is England and Wales. The 2021 counts are the Ethnicity facts and figures table for the 19 tick-boxes and sum to 59,597,578. The 2011 counts are the Census 2011 ethnic-group totals and sum to 56,075,912. Categories with the same id are joined. Roma exists only in 2021 and is not joined. The page says the tick-boxes changed.

**South Africa.** The categories are the census population groups: Black African, Coloured, Indian/Asian, White, and Other. They are not a classification of language or of groups within the Black African population. The 2022 counts of the specified groups are Table 3.3 of Census 2022 in Brief and sum to 61,988,316. The published population is 62,027,503. Unspecified is stored as 39,187. The brief's note of 39,188 unspecified is one person higher; the one-person gap is described on the country page by using the residual that matches the published population. The 2011 group counts in common Stats SA tables sum to 51,770,561 against a published total of 51,770,560. Other is stored as 280,453 so the file matches the published total. Census 2022 has been discussed publicly for coverage error. populon.org still shows the published census counts and says so.

**Malaysia.** DOSM's 2020 key findings give citizens as Bumiputera 20,649,533, Chinese 6,892,367, Indian 1,998,778, and Others 215,637, and non-citizens as 2,691,070. Malay 16,912,998 and Other Bumiputera 3,736,535 are stored separately and sum to the Bumiputera total. Bumiputera is the office's parent category. It is not treated as one ethnic group, and it is not split further in this version beyond Malay and Other Bumiputera.

**Canada.** The chart is the 2021 visible-minority variable, not ethnic origin. Ethnic origin allows more than one answer and does not sum to the population. "Not a visible minority" includes people identified as White and Indigenous peoples. It is labeled as a residual of the Employment Equity Act classification. Components are randomly rounded and sum to 36,328,475, five below the published private-household total of 36,328,480. The page prints that gap. It is inside the 2% rule.

**Australia.** Ancestry is multiple-response, so it is not the chart. Indigenous status partitions the census count into Aboriginal and/or Torres Strait Islander, Non-Indigenous, and Not stated. Non-Indigenous is the complement of that question, not an ancestry. The 2021 QuickStats components sum to 25,422,789, one above the often-cited census count of 25,422,788, from ABS perturbation. The file uses the component sum. The three published Indigenous subcounts sum to 812,730, two above 812,728; they are shown as detail and are not added again. The 2016 and 2021 questions are joined. The page says ABS attributes part of the increase to changes in identification.

**Singapore.** Residents in 2020 are Chinese 3,006,769, Malay 545,498, Indian 362,274, and Other 129,669, summing to 4,044,210. The 2010 figures were published in thousands. Other is stored as 125,700 rather than 125,800 so the rounded parts match the published resident total of 3,771,700. The four categories are joined.

**Ireland.** Census 2022 Profile 5 counts for the updated ethnic-group question sum to 5,084,879 usually resident and present people. The question changed in 2022, so 2016 is not joined. Not stated is kept.

**New Zealand.** Stats NZ level 1 counts for the 2023 Census are European 3,383,742, Māori 887,493, Asian 861,576, Pacific Peoples 442,632, MELAA 92,760, and Other 56,133. The denominator is the census usually resident population of 4,993,923. The counts sum to more than that population. The composition is `multiple_response`. The 2% partition rule is not applied. Stats NZ imputed some missing responses. The 2018 wave is not loaded.

**Kazakhstan.** The Bureau reported 19,186,015 people, Kazakhs at 70.4%, and Russians at 15.5%. The detailed integers are a census composition that sums to 19,186,015. They were checked by that summation and by the Bureau's published percentages, not by re-downloading the census PDF in this build. The residual of 264,111 is other listed groups plus people recorded as another nationality or as nationality not stated. The Bureau's remarks of about 24.8 thousand not stated and about 47.5 thousand "other nationality" sit inside that residual. They are not the whole residual.

**Vietnam.** The GSO preliminary release gives 82,085,729 Kinh and 14,123,255 other ethnic groups, summing to 96,208,984. The completed volume's group-by-group list was not re-extracted, so it is not in the file. A circulating Kinh count of 82,085,826 is not used, because the preliminary pair is the one that sums to the published population.

**Estonia.** The 2021 register-based census ethnic-nationality table gives Estonian 914,896, Russian 322,700, other 85,333, and unknown 7,139, summing to 1,330,068. Ethnic nationality is not citizenship and is not the native-origin / foreign-origin classification, which is a different Statistics Estonia variable and is not used.

**Fiji.** The 2007 census counts are iTaukei 475,739, Indo-Fijian 313,798, and Other 47,734, summing to 837,271. The Government Statistician said on 5 March 2018 that the 2017 census could not release ethnicity. The 2017 population of 884,887 is stored without an ethnic wave. The 2007 wave is marked outdated because the reference year is more than 15 years before 29 October 2026.

**Israel.** The end-2023 CBS estimate, in the form published to three decimal millions, is Jews 7.208 million, Arabs 2.080 million, and Others 0.554 million, summing to 9.842 million. These are population-register groups, not a census ethnic question. The type is `official_estimate`. Others are mainly non-Arab Christians and people not classified by religion.

**Mexico.** The 2020 census population is 126,014,024. The file stores two indicators that overlap: self-identified indigenous people aged 3 and older, 23,232,391, and self-identified Afro-Mexican or Afro-descendant people, 2,576,213. They are not added. Shares use the total population as a common denominator. INEGI's indigenous rate is often computed on the population aged 3 and older, which is higher than the share of the whole population. The country page says that.

**France, Germany, the Netherlands, India, Japan.** No waves. The notes say which concepts exist and why they are not entered as ethnicity.

## Comparison rule

`comparable_series` is a list of blocks. Each block is a list of wave indexes. A line is drawn only when a group id occurs in at least two waves of one block. The same id outside any block is not joined. Groups without an id are not joined. This is tested in `test/join.test.js`.

If a joined wave has `question_change`, or the country has `comparison_caveat`, the chart says the question or the coding changed.

## Map, staleness, and the 2% rule

The map shade is the share of the largest category in the latest ethnic wave, using that wave's total as the denominator. The coastline is Natural Earth 110m. Singapore is absent from that file, so the build draws a mark at 1.35°N, 103.82°E with the same projection. For a multiple-response census, that share is the share of people who reported the category, not a part of a 100% stack. Countries with `data_status: no_official` are hatched. Countries not on populon.org stay empty. The tooltip names the category and the year.

A reference year of 2010 or earlier is marked outdated. The review date is 29 October 2026, so the cut is "more than 15 years."

For `composition: partition`, group counts must be within 2% of the wave total or the build fails. The page also prints the exact gap when it is not zero. Multiple-response and overlapping files are exempt from that test and say why.

## How to add a country

1. Add an object to `scripts/catalog.mjs`. Use the office's categories. Do not map them onto another country's list.
2. Set `composition` to `partition`, `multiple_response`, `overlapping`, or `none`.
3. Leave `comparable_series` empty unless the source itself compares those waves.
4. Run `npm test`, `npm run build`, and `npm run validate`.
5. Add a row to `docs/validation.md` saying what page was opened and what did not match.

The build writes `data/{iso3}.json`, `data/index.json`, `src/generated/atlas.js`, and the HTML table.