A1. Work Week Opening Session: Theoretical Foundations of Critical Quantitative Methods
7/21/25 | 1 PM AST
Presenter(s): David Sul, Ed.D., University of the Virgin Islands
Description The first session of the Work Week will focus on the Critical theoretic foundations of Critical Quantitative methods.
Criticality Statement Sul situates assessment as a force for uplifting communities. His focus on large-scale culturally specific assessment has emerged after a decades-long career as an educator and program evaluator. His work is a descendant of Critical Pedagogy (Freire, 1970), Culturally Relevant Teaching (Ladson-Billings, 1994), and Culturally Responsive Assessment (Hood, 1998). Through the use of modern measurement theory, Sul offers culturally specific assessment as a means to move beyond a dependency on Likert-based measures to produce the numerical values necessary for the conduct of Critical Quantitative research including QuantCrit (Gillborn, et al., 2018) research.
A2. Modeling power: Exploring early childhood education, maternal agency and justifications of harsh discipline using the Cambodian Demographic and Health Survey
7/21/25 | 2 PM AST
Presenter(s): Kelly R. Grace, The University of Texas Medical Branch at Galveston
Description Background
Harsh child discipline is a persistent issue in Cambodia, often rooted in patriarchal norms and the societal acceptance of domestic violence (Miles & Thomas, 2007; UNICEF-Cambodia, 2013). In response, early childhood education (ECE) programs have expanded rapidly, aiming to improve child development and reduce harsh discipline through parenting education and community engagement (Ministry of Education, Youth & Sports, 2014a; Rao & Pearson, 2009). However, feminist scholars critique these programs for reinforcing traditional caregiving roles that may further marginalize women (De Carvalho, 2000; Reay, 1998). Stromquist’s (2015) knowledge empowerment framework offers a counterpoint, suggesting that empowering mothers through education can shift their understanding of power and reduce oppressive practices, including harsh child discipline.
Purpose of the Study
This study explores the relationship between mothers’ contact with ECE programs and their justifications of harsh child discipline, focusing on the mediating roles of maternal agency and involvement in early stimulation activities. The aim is to offer a feminist perspective on mothers’ roles in ECE and child protection.
Methods
Using data from the 2014 Cambodia Demographic and Health Survey (CDHS), the study analyzes responses from 1,809 women aged 18–49 (National Institute of Statistics, Directorate General for Health, and ICF International, 2015). A quasi-experimental design compares mothers with and without ECE program contact. Structural equation modeling (SEM) assesses relationships among maternal agency, early stimulation involvement, and justifications of harsh child discipline, controlling for education, location, and socioeconomic status.
Results
Maternal agency was measured using three latent constructs: access to information, decision-making, and justifications of domestic violence. All loaded significantly onto a second-order agency construct (e.g., decision-making: ß=0.17, SE=0.05, p<.001; access to information: ß=0.41, SE=0.06, p<.001; justifications of domestic violence: ß=0.88, SE=0.01, p<.001). Contact with ECE programs was positively associated with both maternal agency (ß=0.30, SE=0.06, p<.001) and early stimulation involvement (ß=0.36, SE=0.04, p<.001). Maternal agency was negatively associated with justifications of harsh child discipline (ß=-0.89, SE=0.09, p<.001) and significantly mediated the relationship between ECE contact and discipline justifications (ß=-0.26, SE=0.07, p<.001). In contrast, early stimulation involvement did not significantly mediate this relationship (ß=-0.01, SE=0.01, p=.547). The total indirect effect showed that ECE contact was associated with reduced justification of harsh child discipline through maternal agency (ß=-0.17, SE=0.04, p<.001).
Discussion
The findings highlight maternal agency as a key factor in reducing justifications of harsh child discipline. While ECE contact was linked to both agency and stimulation involvement, only agency mediated the relationship with discipline justifications. This challenges assumptions that early stimulation alone improves child outcomes and underscores the need to support mothers beyond caregiving roles. ECE programs should prioritize fostering agency, enabling mothers to challenge patriarchal norms and adopt positive parenting practices.
Implications for Pedagogy, Theory, or Practice
Drawing on feminist theory and Stromquist’s knowledge empowerment framework, this study argues that ECE programs can transform mothers’ understanding of power and reduce oppressive practices. However, the lack of correlation between agency and early stimulation raises questions about how empowerment is operationalized in ECE. Programs should engage mothers as advocates for child protection and support their agency through literacy, access to information, and social networks. Centering mothers in ECE policy and practice offers a holistic approach to advancing both women’s empowerment and child protection.
This study contributes to feminist theory by demonstrating how maternal agency—when supported through ECE—can serve as a tool for liberation rather than oppression. This presentation explores both the challenges and possibilities of integrating feminist theory with SEM using a large national dataset like the CDHS.
References
De Carvalho, M. E. (2000). Rethinking family-school relations: A critique of parental involvement in schooling. Psychology Press.
Miles, G., & Thomas, N. (2007). ‘Don't grind an egg against a stone’—children's rights and violence in Cambodian history and culture. Child Abuse Review, 16(6), 383-400.
Ministry of Education, Youth and Sport (MoEYS) of Cambodia. (2014a). National Action Plan on Early Childhood Development 2014-2018. Phnom Penh.
National Institute of Statistics, Directorate General for Health, and ICF International. (2015). Cambodia Demographic and Health Survey 2014. Phnom Penh, Cambodia, and Rockville, Maryland, USA.
Rao, N., & Pearson, V. (2009). Early childhood care and education in Cambodia. International Journal of Child Care and Education Policy, 3(1), 13.
Reay, D. (1998). Class work: Mothers' involvement in their children's primary schooling. UCL Press.
Stromquist, N. P. (2015). Women's empowerment and education: Linking knowledge to transformative action. European Journal of Education, 50(3), 307-324.
UNICEF-Cambodia. (2013). The Economic Burden of the Health Consequences of Violence Against Children in Cambodia. Phnom Penh, Cambodia.
Criticality Statement Grace believes that society is structured through intersecting layers of oppression that privilege dominant groups. Recognizing her own position within a dominant group, she is committed to research that challenges and disrupts these structures, focusing on reimagining quantitative methodologies—particularly through justice-centered decision-making and tools like structural equation modeling—to expose and transform inequitable systems. Her research applies feminist theory, specifically women's empowerment theory, to examine educational spaces and understand power dynamics in under-explored contexts, aiming for methodological innovation using advanced quantitative modeling. In her study, "Modeling power: Exploring early childhood education, maternal agency and justifications of harsh discipline using the Cambodian Demographic and Health Survey," Grace investigates the relationship between mothers' contact with Early Childhood Education (ECE) programs and their justifications of harsh child discipline in Cambodia, focusing on the mediating roles of maternal agency and involvement in early stimulation activities. Using data from the 2014 Cambodia Demographic and Health Survey (CDHS) and structural equation modeling (SEM), the study found that maternal agency significantly mediated the relationship between ECE contact and reduced justifications of harsh child discipline, challenging assumptions that early stimulation alone improves child outcomes. The findings highlight maternal agency as a key factor in reducing justifications of harsh child discipline and advocate for ECE programs to prioritize fostering agency in mothers, enabling them to challenge patriarchal norms and adopt positive parenting practices, thereby positioning mothers as agents of social change in early childhood education programs.
A3. TBD
7/21/25 | 3 PM AST
Presenter(s): TBD
Description TBD
Criticality Statement
B1. Rethinking "Normality", Eugenics and IQ: Historical Lineages and Critical Alternatives in Educational Assessment
7/22/25 | 1 PM AST
Presenter(s): Benjamin Brumley, West Chester University of Pennsylvania
Description Background
Grading practices and educational and psychological assessment in U.S. have long reflected the ideologies and measurement traditions of the early 20th century, particularly the rise of norm-referenced testing. Rooted in the work of figures such as Alfred Binet and Henry Goddard, early intelligence testing established 'normality' as a central axis around which students were sorted, labeled, and stratified. This framework informed the development of academic aptitude tests, including the SAT, which Carl Brigham advanced in part to serve eugenicist goals. Over time, the logic of ranking, reification of test scores, and the ideal of objective measurement became embedded in educational culture and classroom practice. Yet, as scholars such as Stephen Jay Gould and Nicholas Lemann have argued, the use of standardized assessments often masks deep sociopolitical assumptions about ability, merit, and equity. Emerging traditions offer alternatives that challenge the norm-referenced paradigm and call for a more democratic and pedagogically sound approach to student evaluation.
Purpose of the Study
This presentation investigates how grading traditions have been historically constructed and ideologically maintained, with a focus on norm-referenced assessment as both a technical and political practice. It poses the following research questions: (1) How have norm-referenced traditions shaped grading in K-12 and higher education? and (2) What theoretical and historical critiques exist regarding the foundations of such testing models?
Methods
The study utilizes a critical historical and theoretical framework to analyze the evolution of grading systems in American education. Archival material (e.g., early intelligence testing manuals, university grading policies), foundational texts in educational psychology, and contemporary critical scholarship are triangulated to examine the epistemological and sociopolitical assumptions of grading models.
Discussion
One limitation of this study is its theoretical nature; it does not draw from empirical classroom data but rather focuses on a conceptual reexamination of educational traditions. However, its significance lies in illuminating how dominant grading models, often taken as neutral numbers, are embedded in broader ideological commitments. The findings suggest that the persistence of bell curve grading, item discrimination logic, and deficit-based models of ability is less about pedagogical efficacy and more about inherited assumptions from eugenics, psychometrics, and hierarchical social ordering. These ideologies continue to shape student outcomes, often disproportionately harming students from historically marginalized backgrounds.
Implications for Pedagogy, Theory, and Practice
The implications of this work are threefold. First, pedagogically, educators are encouraged to critically reflect on their assessment practices and consider formative, student-centered approaches that emphasize growth over ranking. Second, theoretically, the study contributes to critical assessment scholarship by foregrounding how grading is never ideologically neutral. It argues for viewing assessment as a site of ethical decision-making and sociopolitical struggle. Third, in practice, institutions should question their reliance on inherited grading systems and instead promote professional development in alternative models such as assessment for learning and anti-deficit evaluation frameworks. By doing so, educators can help dismantle micro-fascist structures in classrooms and foster more equitable, inclusive learning environments.
Keywords: norm-referenced testing, assessment history, grading practices, eugenics, bell curve grading.
Criticality Statement Brumley approaches research as a critical quantitative researcher, interrogating how standardized assessments, particularly those with eugenic legacies, have historically reinforced educational inequities under the guise of objectivity. His work focuses on exposing and mitigating bias in assessment, emphasizing that quantitative methods must be historically informed and ethically deployed to challenge, rather than reproduce, systems of exclusion. Philosophically, his work is influenced by Frankfurt School philosophers like Wilhelm Reich and Hannah Arendt, whose concepts inform democratic assessment practices that resist "micro-fascism" by redistributing power and centering student agency. His current research, "Rethinking 'Normality', Eugenics and IQ: Historical Lineages and Critical Alternatives in Educational Assessment," investigates how grading traditions have been historically constructed and ideologically maintained, with a focus on norm-referenced assessment as both a technical and political practice. Through a critical historical and theoretical framework, analyzing archival material and contemporary scholarship, the study argues that dominant grading models, often perceived as neutral, are embedded in broader ideological commitments inherited from eugenics and psychometrics, disproportionately harming historically marginalized students. The implications of his work are threefold: pedagogically, encouraging critical reflection on assessment practices; theoretically, contributing to critical assessment scholarship by foregrounding the ideological nature of grading; and practically, urging institutions to question inherited grading systems and promote alternative, anti-deficit evaluation frameworks to foster equitable learning environments.
B2. TBD
7/22/25 | 2 PM AST
Presenter(s): TBD
Description TBD
Criticality Statement
B3. How Different Could It Be?” Exploring Instructional Quality Effects on “Algebra for All
7/22/25 | 3 PM AST
Presenter(s): Jialu Fan, University of Minnesota, Twin Cities
Description Many scholarships have stated that algebra is not only hard to learn but difficult to teach well (Wojongan et al., 2023). While a number of studies have explored this topic (e.g., Ladson-Billings et al.,1994; Sleeter, 2001; Zeichner 2002; Cochran-Smith, 2004), more investigation is needed to ascertain how specific instructional practices impact algebra achievement.
This research is situated in institutional and structural racism. Institutional racism encompasses discrimination, unjust policies, and inequitable opportunities for the underrepresented and racially minoritized (URM) groups immortalized by institutions (institutional actors) such as schools, cooperative organizations, etc. (Zambrana et al., 2017; McGee, 2020). For example, economically disadvantaged minority students in urban schools are offered low-quality mathematics instruction (Eisenhart et al, 1993; Silver & Stein, 1996; Lee, 2012). From a broader lens, structural racism manifests through implicit or explicit rules within interrelated institutions, creating a system that upholds White supremacy and allows racism to reinvent and persist (Gee & Hicken, 2021). For example, although each school and its teachers in classrooms seem to hold their own beliefs and perspectives regarding algebra education, the racialized rules are perpetuated often through implementing certain instructional practices for certain racial populations.
RQ1: To what extent does instructional quality correlate with students’ achievement in eighth-grade algebra?
RQ1.1: How do teacher’s specific pedagogical practices (i.e., dimensions of instructional quality) affect student achievement in eighth-grade algebra?
RQ1.2 What are the effects of teacher’s specific pedagogical practices (i.e., dimensions of instructional quality) on students with different racial and socioeconomic statuses in eighth-grade algebra achievement?
RQ1.3: To what extent do seventh-grade MCA scores moderate the instructional quality--eighth-grade algebra achievement relationship?
This research study employs hierarchical linear modeling to examine educational data that are predominantly nested and structured hierarchically (Hox, 1998; Raudenbush & Bryk, 2002; Anderson, 2012). The variables of race, socioeconomic status, seventh-grade MCA score, and eighth-grade MCA score are measured at the student level, while instructional quality in seven dimensions is assessed at the classroom (teacher) level. The dependent variable is the eighth-grade MCA score, which measures student achievement in eighth-grade algebra. All variables, except for instructional quality, are provided as raw data by the Minnesota Department of Education. The survey instrument is developed based on a) the seven-dimensional instructional quality framework suggested by Mu et al. (2022), b) the conceptual and operational indicators for assessing instructional quality outlined by Praetorius et al. (2018) and Mu et al. (2022) , c) informal discussion sessions with current and previous math teacher candidate supervisors at the Teacher Education Research Group (TERG) hosted by the Department of Curriculum and Instruction.
Previous research (Teig & Luoto, 2024) exploring the relationship between instructional quality and student achievement in mathematics and science has yielded inconsistent results among the dimensions. Further, there are few large-scale studies examining algebra instruction (Litke, 2020). As a result, the field of mathematics education still lacks a sufficient understanding of the nature of algebra pedagogy and how instructional practices that support students' learning of algebra are implemented in the classroom (Litke, 2020). This study addresses gaps in instructional quality literature and adds to existing scholarship by exploring how specific instructional practices affect student achievement in eighth grade algebra.
This study employs critical lenses and hierarchical linear modeling to examine how specific pedagogical practices in eighth-grade algebra classrooms support the learning of underrepresented and racially minoritized (URM) students, offering new perspectives to interrogate the "Algebra for All" policy. Additionally, by building on the instructional framework by Mu et al. (2022) and adapting the empirical instrument, this study aims to contribute to the emerging understanding of instructional quality in teaching algebra.
Criticality Statement Fan operates from a broad worldview of pragmatism, focusing on research inquiry, practical data collection, and embracing multiple perspectives. Their work is deeply informed by Critical Race Theory (CRT), which they apply to uncover institutional and structural racism in large-scale investigations of instructional quality and to understand the concept of invisibility in the learning experiences of underrepresented and racially minoritized (URM) students. Fan's research, titled "How Different Could It Be? Exploring Instructional Quality Effects on 'Algebra for All'," addresses the challenge of teaching and learning algebra effectively, especially given that economically disadvantaged minority students often receive low-quality mathematics instruction due to institutional and structural racism. The study investigates how instructional quality correlates with 8th-grade algebra achievement, specifically examining the impact of pedagogical practices on student achievement across different racial and socioeconomic statuses, and how prior academic scores moderate this relationship. Employing Hierarchical Linear Modeling (HLM) on nested educational data, Fan's work aims to fill gaps in instructional quality literature by exploring how specific practices affect 8th-grade algebra achievement, particularly for URM students. This research offers new perspectives to interrogate the "Algebra for All" policy and contributes to advanced quantitative methods by rigorously combining HLM with critical perspectives like CRT to examine equity implications in mathematics education.
C1. What and Why Demographic Data are Collected: A Landscape Analysis of Research and Practice
7/23/25 | 1 PM AST
Presenter(s): Esther Nolton, Everstead Strategies and Tom Workman, American Institutes for Research
Description There is a growing interest and need for organizations to capture demographic information to help characterize engagements, conduct research, or inform strategic decisions. Without this information, it can be difficult to understand the various needs, experiences, perspectives, and outcomes across diverse populations. There are many ways in which institutions collect this information but it is unclear in what ways organizations collect these data, for what reasons, and how they are being used to make their work more relevant and useful to the groups they serve.
This landscape analysis will help outline the varied ways in which demographic data are collected and the purpose of those efforts that inform methodological decisions. It will include an analysis of published and grey literature documenting the frameworks and practices of various organizations to collect demographic information. Key informant interviews and/or focus groups will be conducted with demographers, researchers, program administrators, survey methodologists, and other professionals who study and/or make methodological decisions regarding demographic data collection to further understand the various ways and reasons demographic data are collected in different sectors, contexts, domains, and circumstances.
A preliminary analysis of literature was completed but primary data collection is still in progress. A subset of published and grey literature were abstracted for thematic analysis and discussed among a panel of experts. In general, findings revealed an evolving set of motivations, models, practices, and gaps in knowledge and practice. No formal standards exist to guide organizations wishing to collect inclusive demographic data for administrative purposes. Researchers found varying, and at times conflicting, purposes and contexts for revisions to demographic response items specifically with the demographic characteristics of race, ethnicity, gender identification, sexual orientation, and disability.
Like all forms of data collection, demographic data collection must be purposeful and useful to serve the anticipated needs of the effort to capture those data. Organizations wishing to collect demographic data must still address several key considerations surrounding disaggregation, representation, self-identification, and the protection of privacy and weigh various practices based on the context, purpose, and use of the data being collected. This landscape analysis will provide some helpful considerations as researchers and practitioners of demographic data collection seek to capture data that are useful and relevant to their needs.
Criticality Statement Nolton grounds her commitment to empowerment, enfranchisement, and engagement in her identity as a daughter of Taiwanese immigrants, having experienced misconceptions about her culture and the consequences of not being accounted for in public programs and policies. With over 15 years in research and evaluation, she applies a culturally responsive and humble lens, emphasizing the inclusion and engagement of individuals and groups affected by programs and their evaluations. Her research, titled "What and Why Demographic Data are Collected: A Landscape Analysis of Research and Practice," addresses the growing need for organizations to capture demographic information to characterize engagements, conduct research, and inform strategic decisions, noting the lack of clarity on how such data are collected, for what reasons, and how they are used to serve diverse groups. This landscape analysis aims to outline the varied ways demographic data are collected and their purposes, informing methodological decisions through an analysis of published and grey literature, supplemented by key informant interviews and/or focus groups. Preliminary findings reveal evolving motivations, models, practices, and gaps, with no formal standards for inclusive demographic data collection and varying purposes for revisions to demographic characteristics like race, ethnicity, gender identification, sexual orientation, and disability. The study emphasizes that demographic data collection must be purposeful, considering disaggregation, representation, self-identification, and privacy, ultimately providing practitioners with options and researchers with a foundational framework for future research on demographic data collection.
C2. Critical Analysis of School Evaluation Data from the Knowledge and Human Development Authority (KHDA) Dubai
7/23/25 | 2 PM AST
Presenter(s): Emily Winchip, Zayed University Abu Dhabi
Description This research aims to analyze the quality of publicly reported data of private school quality in Dubai. School evaluations are conducted annually by the Dubai School Inspection Bureau as an accountability measure for private schools through the Knowledge and Human Development Authority (KHDA). The schools are rated on up to 102 items and assigned an overall rating. Significant importance is placed on the Overall Ratings of a school as it determines the allowed percent rise in school fees in a system of primarily for-profit private schools. The main questions of this research are: Does the school evaluation data fit a Rasch model? Which items do not fit the model? Which items are related to the overall rating, and which are not? What can this analysis tell us about the limits of the usefulness of the data?
The items were analyzed with Rasch analysis to explore the unidimensionality of the scale, the quality of the items to contribute to the full scale, and the items’ relation to the overall rating.
The UAE school system encompasses a wide variety of curricula, costs and offerings by schools where a majority of students are expatriates, not served by the government system, and educated in private schools, often operated for-profit. The UAE system fits a trend noted by Shafiq (2011) that Middle East countries have tended to take an incentive and accountability-based approach to overseeing education, focusing on curricular autonomy, competition between schools, and publicly available performance data intended to inform parental choice. Research about the UAE has noted the unique approach to autonomy with school inspection and has found the intended effects of inspection have been mixed and come along with side effects. Alkutich and Abukari (2018) found that Dubai teachers felt that school evaluations had informed teaching and learning but that the reports often felt superficial to the teachers and did not clearly lead to productive responses improving teachers’ work. Other research conducted in the UAE found the focus on external accountability of school evaluation to be part of a set of factors that inhibit teacher collaboration, potentially harming school improvement in the UAE (Ibrahim, 2020). The UAE is potentially part of a broader trend that education has moved away from a means to national improvement and towards a focus on individual advancement through competitive comparison between educational institutions (Robertson, 2012).
This research is a secondary data analysis of the KHDA school evaluations including data from 200 schools in Dubai that had been inspected during the 2022-2023 school year. Each school in Dubai was evaluated on up to 102 items. The data, publicly available on the KHDA website, was coded as numbers and analyzed through Rasch analysis.
The analysis of the school evaluation items found that a modified scale including the Overall Rating has some robustness as a scale. However, the modified scale was only found to be robust when it excluded most of the Arabic language and Islamic studies items. With these findings, questions about the validity of the scale and the usefulness of the data can be addressed.
Analyzing the validity of a scale can mean looking at a large variety of questions of the meaning of the scale including how well a scale describes a construct, how a scale covers the important content of a construct, the statistical power of a scale to make statements about variation, the sensitivity of measures to changes, reliability of a scale across participants, the internal consistency of a scale, and generalizability across participants (Cherryholmes, 1988). This research is a basic construct validity analysis, investigating the unidimensionality of the scale and the internal validity of the scale. The findings show that the school evaluation scale includes a threat to content validity that the modified scale is robust only when it excludes items about required subjects in the UAE school system. It is notable that the tested scale including Arabic language and Islamic studies items demonstrated clearly that the overall rating was not related to these items. As whole, the items do not define a single construct and are not well represented by the overall rating.
References
AlKutich, M., & Abukari, A. (2018). Examining the benefit of school inspection on teaching and learning: a case study of Dubai private schools. Journal of Education and Practice, 9(5).
Cherryholmes, C. H. (1988). Construct Validity and the Discourses of Research. American Journal of Education, 96(3), 421–457.
Ibrahim, A. (2020). What hurts or helps teacher collaboration? Evidence from UAE schools. PROSPECTS. https://doi.org/10.1007/s11125-019-09459-9
Robertson, S., Mundy, K., & Verger, A. (2012). Public private partnerships in education: New actors and modes of governance in a globalizing world. Edward Elgar Publishing.
Shafiq, M. (2011). Do School Incentives and Accountability Measures Improve Skills in the Middle East and North Africa? The Cases of Jordan and Tunisia. Review of Middle East Economics and Finance, 7(2), 1-28. https://doi.org/10.2202/1475-3693.1279
Criticality Statement Winchip brings a perspective shaped by her experience as a former teacher in diverse school settings and her training as a quantitative analyst, which led her to question the validity of high-stakes school evaluation methods. Her research focuses on educational measurement, specifically investigating how it affects education policy and the experiences of teachers, with an aim to develop the field of critical educational measurement by analyzing the ethics, data quality, and consequences of data use in education. She seeks to bridge the gap between numerical data and the lived experiences of people in schools, noting that numbers are often prioritized over human experiences. Her current research, "Critical Analysis of School Evaluation Data from the Knowledge and Human Development Authority (KHDA) Dubai," aims to analyze the quality of publicly reported data on private school quality in Dubai. This study uses Rasch analysis to explore the unidimensionality and internal validity of the school evaluation scale, and how items relate to the overall rating. The analysis of data from 200 schools inspected in 2022-2023 revealed that a modified scale, excluding most Arabic language and Islamic studies items, showed some robustness, but the overall rating was not clearly related to these required subjects. This finding raises questions about the validity of the scale and the usefulness of the data, highlighting a threat to content validity. Winchip's research contributes a procedure for analyzing the quality of high-stakes school evaluations, providing a model for secondary data analysis to understand school evaluation practices globally.
C3. TBD
7/23/25 | 3 PM AST
Presenter(s): TBD
Description TBD
Criticality Statement
D1. Global Majority Arts Administrators, Arts Educators, & Artists in the U. S.: What are their Cultural Policy Priorities?
7/24/25 | 1 PM AST
Presenter(s): antonio c. cuyler, University of Michigan
Description Extant literature (Besana and Esposito 2022; Caust 2019; Elpus 2016; Novak-Leonard and Skaggs 2021; Taylor 2022) has not centered or operationalized global majority arts administrators, arts educators, and artists’ community cultural wealth (Dragićević Šešić 2024; Yosso 2015) to inform the development of a national arts advocacy agenda based on their policy priorities. By centering global majority creatives in this study, we reveal new knowledge critical to cultivating a thriving anti-racist creative sector in the U. S., specifically as it relates to the development of a national arts advocacy agenda. In addition, this study’s results advance extant knowledge by uncovering global majority creatives’ policy preferences when advocating for arts and culture which aligns with the Arts Administrators of Color Network’s (AAC) four-year strategic plan.
We investigated two research questions in this descriptive research study. First, what policy issues matter most to global majority arts administrators, arts educators, and artists when advocating for arts and culture? Second, do differences exist in the prioritization of policy issues based on the intersectional demographic profiles of global majority arts administrators, artists, and arts educators?
CRT theorized that white Americans structurally and systemically built racial caste into U. S. society to produce observable negative disparate outcomes for Black Americans based on 246 years of slavery, followed by more than a century of racial terror and segregation after the U. S. civil war, and before civil rights legislation in 1964, 1965, and 1968. CRT inspired Asian (AsianCrit), Hispanic/Latine (LatCrit), Indigenous (TribalCrit), and Jewish (HebCrit) scholars in developing critical race theories that enabled their understanding of the ways that they, too, experience negative disparate outcomes based on the U. S.’ racial caste system. Crenshaw (1989) articulated intersectionality to theorize the ways that race intersects with non-racial social identities such as affectional orientation (QueerCrit), class (ClassCrit), disability (DisCrit), and gender (FemCrit) to amplify the impacts of observable negative disparate outcomes by two, three, and four fold. Given these theories, we assume that global majority arts administrators, arts educators, and artists have embodied and lived racialized experiences because of enduring racism in U. S. society. Racism has forced global majority creatives to develop community cultural wealth (Yosso, 2015) to lead a flourishing career in the creative sector.
To analyze the data, we used descriptive statistics to report means and sums using a critical race theories analytic framework. To answer the second research question, we used Crenshaw’s (1989) intersectionality (ClassCrit, DisCrit, FemCrit, and QueerCrit) as the analytic framework revealing some differences in the prioritization of policy issues based on global majority arts administrators, arts educators, and artists’ demographic profiles.
Across two iterations of data collection, this study revealed that global majority creatives prioritized nine policy issues that will inform the development of a national arts advocacy agenda. These policies included: federal funding for arts education, federal funding for the National Endowment for the Arts (NEA), city/county funding for culture, state funding for culture, protecting affirmative action, access, diversity, equity, and inclusion; tax fairness for artists, access to lifelong arts education, federal funding for the National Endowment for the Humanities (NEH), and affordable housing. While we expected respondents to rank national and subnational government funding highly, the rankings surprised us because they took precedence over policies such as climate change, democracy, Medicare for all, student loan debt relief, and universal basic income.
Given that a kakistocratic administration has given rise to the politics of cruelty, the most important implication for this study is that it manifests an evidence-based approach to policy advocacy that can improve the lives of those caste in U. S. society as the least among us.
Criticality Statement Cuyler grounds his research in critical theories like Afrofuturism, Critical Race Theory (CRT), and Intersectionality, asserting that all research inherently reflects human biases and subjectivities. As an anti-positivist interpretivist, he challenges the concept of "objectivity" as a characteristic of White supremacy culture, advocating for its scholarly interrogation. His research agenda focuses on how the creative sector can ensure and protect the creative justice of historically marginalized individuals, integrating various fields to explore access to arts and culture. His study, "Global Majority Arts Administrators, Arts Educators, & Artists in the U. S.: What are their Cultural Policy Priorities?", addresses a gap in literature by centering the community cultural wealth of global majority arts administrators, arts educators, and artists to inform a national arts advocacy agenda. This descriptive study, utilizing CRT and intersectionality, identified nine key policy issues prioritized by global majority creatives, including federal and state funding for arts and culture, federal funding for arts education, protecting affirmative action, and affordable housing, contributing significantly to Critical Quantitative (QuantCrit) methods by empowering marginalized voices in policy advocacy within the U.S. creative sector.
D2. Reframing Perception as Structure: A Critical Quantitative Analysis of Schooling, Risk, and Postsecondary Access in the NLSY97
7/24/25 | 2 PM AST
Presenter(s): Catherina Villafuerte, University of Connecticut
Description This study provides a critical reconceptualization of student perception data, reframing these measures as structurally embedded indicators of institutional opportunity and systemic inequity. Using data from the National Longitudinal Survey of Youth 1997 (NLSY97), the analysis investigates how adolescent perceptions of school climate, peer norms, and structural risks, as reported in 1997, predict subsequent postsecondary enrollment outcomes by 2002. Contrary to dominant approaches that position student perception as merely subjective or attitudinal, this study employs Critical Race Theory (CRT), Intersectionality, and Structural Theories of Schooling to argue that perceptions reflect institutional realities shaped by power, race, and social positionality. Through this approach, perception data are treated not as individual-level psychological constructs but as collective testimonies reflecting systemic educational conditions.
The study is guided by two primary research questions. First, it seeks to determine to what extent students' perceptions of school climate, peer norms, and structural risk predict differential patterns of postsecondary enrollment. Second, it explores how these predictive relationships vary according to students' intersecting racial, ethnic, and gender identities. The analytic sample comprises over 8,500 respondents who provided complete data on the selected perception constructs and demographic variables. Composite indices measuring school climate, peer norms, and structural risk were created through exploratory factor analyses and standardized to facilitate multinomial logistic regression modeling. The dependent variable was categorically defined, differentiating full-time, part-time, ambiguous forms of postsecondary enrollment, and non-enrollment.
Results substantiate the conceptual reframing proposed in the study, confirming that adolescent perceptions significantly predict postsecondary enrollment trajectories. Students who perceived more supportive school climates and affirming peer norms experienced increased odds of full-time enrollment in four-year colleges. Conversely, perceptions of heightened structural risk were significantly associated with decreased access to these educational pathways. Notably, these relationships were not uniform across social identities. The association between school climate perceptions and college enrollment was moderated by gender, with female students experiencing relatively diminished protective effects. Additionally, peer norms held greater predictive significance for mixed-race and non-Black, non-Hispanic students, suggesting variations in how peer environments intersect with institutional structures. The detrimental impact of structural risk perceptions was particularly pronounced among Black students, reinforcing the racialized dimension of systemic educational inequity.
Although the analytic scope of this investigation focuses on a single outcome year, its contributions extend beyond empirical modeling to encompass methodological and theoretical advances within critical quantitative scholarship. By intentionally reframing student perception data through justice-oriented epistemologies, this research challenges traditional quantitative approaches that overlook systemic and structural explanations in favor of individual-level narratives. Identity variables, typically treated as covariates or statistical controls, are repositioned as critical sites for structural examination, allowing for richer interpretations of how race, ethnicity, and gender interact within educational contexts.
The study carries significant implications for educational policy, institutional practice, and methodological frameworks. Educational systems should integrate student perception measures into accountability and reform agendas as structural diagnostics capable of illuminating inequities often obscured by standardized metrics. In terms of theory, the research contributes substantively to critical quantitative methodologies by demonstrating practical strategies for aligning empirical rigor with justice-centered frameworks. Pedagogically, the study offers a detailed methodological exemplar, guiding scholars toward meaningful integration of critical theory into quantitative analyses. Ultimately, this research affirms the value of centering student voices as structurally meaningful indicators of institutional conditions, calling for a paradigmatic shift in how educational equity and access are conceptualized, measured, and addressed.
Criticality Statement Villafuerte approaches research from a justice-oriented epistemological stance, viewing knowledge production as deeply embedded in systems of power, colonial legacies, and racial capitalism. She rejects the neutrality of educational institutions, instead examining how they reproduce structural inequities and silence marginalized voices, prioritizing methodologies that center lived experience and dismantle systems of exclusion. Her research is grounded in critical theory, drawing from Critical Race Theory, decolonial thought, and structural analyses to expose how dominant ideologies perpetuate racial and epistemic hierarchies within research design and policy. In her study, "Reframing Perception as Structure: A Critical Quantitative Analysis of Schooling, Risk, and Postsecondary Access in the NLSY97," Villafuerte critically reconceptualizes student perception data from the National Longitudinal Survey of Youth 1997 (NLSY97), arguing that these perceptions reflect institutional realities shaped by power, race, and social positionality. The analysis, involving over 8,500 respondents, found that adolescent perceptions of school climate, peer norms, and structural risks significantly predicted postsecondary enrollment trajectories by 2002. Specifically, students who perceived more supportive school climates and affirmed peer norms had increased odds of full-time enrollment in four-year colleges. At the same time, perceptions of heightened structural risk were associated with decreased access. Notably, these relationships varied by identity: the protective effects of school climate perceptions were diminished for female students, peer norms held greater predictive significance for mixed-race and non-Black, non-Hispanic students, and the detrimental impact of structural risk perceptions was particularly pronounced among Black students. This research advances critical quantitative scholarship by challenging traditional approaches that overlook systemic explanations, repositioning identity variables as central structural dimensions, and advocating for student voices as meaningful indicators of institutional conditions.
D3. "Always Been That Way" A conversation on intentional and ethical selection of within- and, or, between-group quantitative analyses
7/24/25 | 3 PM AST
Presenter(s): Shannon Casey, Institute for Clinical and Translational Research at the University of Wisconsin - Madison
Description With every build or reiteration of a healthcare or education program, program staff, evaluators, and methodologists evaluate the design of the program for audiences that vary on many characteristics. Though we consider intersectionality, which we all understand intuitively and in practice, it can be particularly challenging to understand and evaluate the unique effects of a program designed for a group that shares a characteristic (e.g., sufferers of a certain illness, trainees in a specific program, users of a particular service) but also variable on other important characteristics (e.g., age, ability status, geography, cultural belonging, trauma background). Fifty years or more of research by statistical and cultural experts (e.g., Janet Helms 1992; Matusmoto & Van de Vijver, 2010), has identified conditions under which comparisons across groups or deep dives within groups reflect the nature of the research question.
Too often, working in high-paced and high-stakes environments, evaluation defaults to what has been done before. While a reasonable approach, we may miss important reflection on possible harms to communities when groups are compared, or possible missed opportunities when groups are not compared. How do we make those decisions about comparing across or within groups? How do those comparisons reflect in the reports we write or the graphics we share?
This session will introduce key questions for reflection as a research team as you build and design your evaluation research, including methodological considerations (e.g., subgroup sample size, measurement equivalent, confounds) and group differences (e.g., lingualism, identity, group norms). We will evaluate and observe key points made by experts who study and advocate for ""within"" or ""between"" group analyses, looking critically at this impactful and sometimes rushed decision. Multiple examples from personal experience doing research with diverse groups and from the literature, across the research trajectory from study design to reporting, will be shared. We will discuss the compelling new, but sometimes fraught, territory of Artificial Intelligence and Language Learning Models. We will work towards identifying which approaches fit better for our own unique research questions. We will consider the risks and rewards of doing both between and within group comparisons in the same design. Attendees will be invited into small groups to discuss the challenges inherent in their own work and commit to some small improvement aimed at continuous quality improvement for their own teams.
We all know the ""right"" answer is not ""it's always been that way"" but rather a thoughtful and connected collaborative conversation of how we maximize our ability to make inference and also maximize our safety for all who participate. You are invited to explore openly but critically your own professional approach, for which there may not be a “right answer,” but rather the most appropriate answer for the critical impact you are wishing to have.
References
Helms, J. E. (1992). Why is there no study of cultural equivalence in standardized cognitive ability testing?. American psychologist, 47(9), 1083.
Matsumoto, D., & Van de Vijver, F. J. (Eds.). (2010). Cross-cultural research methods in psychology. Cambridge University Press.
Criticality Statement Casey believes researchers have an obligation to address diversity and culture to support equity, advocating for comprehensive, sustainable solutions through interdisciplinary collaboration and sophisticated methodological knowledge. Her research critically examines systems that perpetuate oppression, collaborating with healthcare, education, and social researchers to promote community engagement, partnership building, and participatory, collaborative methodologies that reduce inequities. In her presentation, "Always Been That Way: A conversation on intentional and ethical selection of within- and, or, between-group quantitative analyses," Casey addresses the common practice in program evaluation of defaulting to past methods without critically considering the potential harms or missed opportunities when comparing or not comparing diverse groups. The session aims to guide research teams in making intentional decisions about within- or between-group analyses, discussing methodological considerations, group differences, and the role of Artificial Intelligence, while encouraging personal and collective reflection to prevent systematic harms when cultural confounds are overlooked.
E1. From Access to Equity: Mapping Policy Solutions for Anti-Ableism in Higher Education
7/25/25 | 1 PM AST
Presenter(s): Jessica Lopez, Arizona State University STEM Program Evaluation Lab
Description While disabled students represent approximately 19% of the undergraduate population, their graduation rate lags nearly 20 percentage points behind their non-disabled peers (NCES, 2023). This disparity is often framed as an individual deficit rather than the result of systemic inequities shaped by policy, institutional design, and epistemic erasure. Dominant frameworks in higher education rely on compliance with the Americans with Disabilities Act (ADA) rather than interrogating the ableism embedded in institutional policies, pedagogies, and structures. Critical Disability Studies (Dolmage, 2017) has made visible the cultural and structural dimensions of ableism, but there is limited research that bridges these insights with large-scale policy analysis. This presentation addresses that gap through a critical, data-driven analysis of disability-related higher education laws across the U.S., using the findings to develop an anti-ableist policy agenda grounded in both empirical evidence and disabled student experience.
Drawing from disability justice principles and critical quantitative frameworks, we sought to recenter disabled student needs and reframe disability policy not as compliance-based, but as a site of cultural transformation. The guiding research questions were: (1) How can an anti-ableist framework improve accessibility and success rates in higher education? (2) What are the most effective policy and curriculum-based strategies for fostering inclusion for disabled students?
Our methodology included a 50-state legal review of publicly available higher education codes, statutes, and administrative policies that explicitly reference disability or accommodations. Each state's data was coded to quantify the number of relevant laws, identify the presence or absence of protections for disabled students, and analyze the thematic focus of each law (e.g., digital accessibility, inclusive programs, faculty training). This dataset was then layered with qualitative insights from a literature review. The combination of content analysis and quantitative state-by-state comparisons enabled us to identify patterns of neglect, innovation, and regional disparity.
Findings revealed that over one-third of U.S. states have zero state-level protections for disabled students in higher education beyond federal requirements. States with stronger protections often had mandates for digital accessibility, emergency planning, or inclusive higher ed programs for students with intellectual disabilities, but lacked mechanisms for accountability, cultural change, or structural reform. Based on these findings, we developed 13 policy recommendations spanning institutional, state, and federal levels.
A limitation of this study is that laws and policies are constantly evolving, and publicly available legislative databases may not capture all relevant measures. However, the broader contribution of this work lies in its integration of critical disability theory with policy mapping and institutional critique. It also contributes to critical quantitative methods by offering a model of how legal data and state policy analysis can support equity-driven agendas.
This research has several implications for practice and pedagogy. First, it provides a concrete framework for disability-inclusive institutional change that goes beyond ADA compliance. Second, it highlights the need to embed anti-ableism into the teaching of higher education policy and governance. Finally, it invites scholars and practitioners to view policy as a site of cultural production, where what is written into law shapes who is recognized, resourced, and respected on campus. In this way, critical policy analysis becomes a tool not only for critique, but for collective redesign.
Criticality Statement Lopez approaches research as a tool for systemic change, driven by a worldview rooted in disability justice that emphasizes interdependence, access, and the lived experiences of marginalized communities. As a disabled Latina scholar and organizer, she believes data should serve those most impacted, translating structural critiques into practical tools for policy and educational reform by interrogating systems for what they exclude and fail to imagine. Her research challenges dominant compliance-based frameworks in higher education policy by interrogating how ableism is embedded in institutional practices, laws, and norms, using research to expose and disrupt these systemic inequities. Drawing from intersectionality, she explores how disability interacts with race, class, and gender in shaping educational access and success, framing disability as a sociopolitical condition. In her presentation, "From Access to Equity: Mapping Policy Solutions for Anti-Ableism in Higher Education," Lopez addresses the disparity where disabled students, approximately 19% of the undergraduate population, lag nearly 20 percentage points behind their non-disabled peers in graduation rates. Through a 50-state legal review of higher education codes and policies, her study revealed that over one-third of U.S. states have zero state-level protections for disabled students beyond federal requirements, and even states with stronger protections often lack accountability mechanisms for cultural or structural reform. Based on these findings, 13 policy recommendations were developed to foster inclusion and improve accessibility. This work advances Critical Quantitative Methods by integrating critical disability theory with policy mapping and institutional critique, offering a framework for disability-inclusive institutional change that moves beyond mere ADA compliance and repositions disabled students as critical knowledge producers within equity-driven evaluation.
E2. Beyond the Black Box: Interrogating the Theoretical Foundations of AI in Performance Management for a More Equitable Future
7/25/25 | 2 PM AST
Presenter(s): Nathalie Salles-Olivier, University of the Virgin Islands
Description Background
The integration of Artificial Intelligence (AI) into performance management (PM) systems is rapidly transforming the modern workplace. While proponents argue that AI can enhance efficiency and objectivity, critical scholars raise concerns about the potential for these systems to perpetuate and amplify existing inequalities, particularly along gender lines. The literature reveals a significant gap in our understanding of the theoretical foundations and worldviews underpinning these AI-driven PM systems. This study, therefore, draws on critical theory, feminist theory, and intersectionality to provide a more nuanced and holistic understanding of this complex issue.
Purpose of the study
The purpose of this study is to critically examine the theoretical foundations of AI-driven performance management and its impact on gender equity in the tech industry. The central research question is: How do the theoretical underpinnings and authorial worldviews of AI-driven performance management systems, as reflected in the literature from 2019 to 2024, contribute to or mitigate gender inequity in the tech industry? This study seeks to move beyond a simplistic "good vs. bad" dichotomy and instead explores the inherent tensions and contradictions within this socio-technical phenomenon.
Methods
This study employs a novel methodological approach centered on the development and application of a bespoke AI assistant for a systematic literature review of scholarly articles, conference proceedings, and industry reports published between 2019 and 2024. This specialized AI is designed to analyze the corpus by: (1) identifying the presence or notable absence of explicit theoretical frameworks and authorial positionality statements (worldviews); (2) mapping the research methodologies utilized; and (3) evaluating the alignment between the stated theories and methods. This AI-driven analysis will be supplemented by a critical bibliometric review to map the geographical and institutional origins of the research, providing context to the dominant paradigms. This innovative, dual-pronged method allows for a nuanced critical discourse analysis of the underlying assumptions shaping the discourse on AI in performance management.
Discussion
The preliminary findings suggest that while there is a growing awareness of the potential for bias in AI, there is a lack of a robust theoretical framework for understanding and addressing these issues from a social justice perspective. The discussion will focus on the limitations of purely technical or legalistic approaches to fairness and the need for a more critical and intersectional lens. It will also explore the "double-edged sword" nature of AI in PM, highlighting both its potential to mitigate certain forms of bias and its risk of creating new, more insidious forms of discrimination.
Implications for pedagogy, theory, or practice
This research has significant implications. For pedagogy, it provides a model for teaching how to build and utilize AI tools for critical inquiry, moving beyond using AI as a simple search tool. For theory, it introduces a new method for conducting theoretical analysis at scale and contributes to a more critical understanding of AI's impact on work. For practice, it offers a framework for designing and implementing AI-driven PM systems that are more equitable and human-centered.
Criticality Statement Salles-Olivier brings a pragmatic and critical worldview shaped by her French-American background, international business experience, and pursuit of a PhD in Creative Leadership at a Caribbean HBCU. She is committed to bridging theory and practice, driven by a desire to promote social justice and equity for women and marginalized groups in the workplace. Her research is situated within critical theory, drawing from critical pedagogy, feminist theory, and intersectionality, to examine the social and ethical implications of technology, particularly AI, in the workplace, aiming to challenge power dynamics and systemic biases in organizational systems like performance management. Her current research, "Beyond the Black Box: Interrogating the Theoretical Foundations of AI in Performance Management for a More Equitable Future," critically examines how the theoretical underpinnings and authorial worldviews of AI-driven performance management systems, as reflected in literature from 2019-2024, contribute to or mitigate gender inequity in the tech industry. Employing a novel methodology that includes a bespoke AI assistant for systematic literature review and critical bibliometric analysis, her preliminary findings suggest a lack of robust theoretical frameworks for addressing AI bias from a social justice perspective. This work has significant implications for pedagogy, theory, and practice by offering a model for using AI tools for critical inquiry, a new method for theoretical analysis at scale, and a framework for designing more equitable and human-centered AI-driven performance management systems.
E3. Work Week Closing Session: A New Direction for the Conduct of Critical Quantitative Reseearch
7/25/25 | 3 PM AST
Presenter(s): David Sul, Ed.D., University of the Virgin Islands
Description The closing session of the Work Week will focus on new directions for the study and practice of Critical Quantitative methods. It will include feedback and commentary from the Work Week participants.
Criticality Statement Sul situates assessment as a force for uplifting communities. His focus on large-scale culturally specific assessment has emerged after a decades-long career as an educator and program evaluator. His work is a descendant of Critical Pedagogy (Freire, 1970), Culturally Relevant Teaching (Ladson-Billings, 1994), and Culturally Responsive Assessment (Hood, 1998). Through the use of modern measurement theory, Sul offers culturally specific assessment as a means to move beyond a dependency on Likert-based measures to produce the numerical values necessary for the conduct of Critical Quantitative research including QuantCrit (Gillborn, et al., 2018) research.
