Commercial agriculture is central to problems of sustainability in food, water, energy and climate change. Appropriate solutions will depend on the effective promotion, monitoring and evaluation of changes in farming practice. Conservation agriculture (CA) is an important example of sustainable intensification and climate-smart agriculture, increasing the productivity and reliability of grain production while reducing agricultural inputs and future climate risks when adopted comprehensively. But to understand its implementation and benefits, researchers often rely on simplified measures of CA adoption (e.g., single proxies, binary measures, broad self assessments, expert estimates). Here we use a national survey of South Africa's commercial grain farmers (n = 441), contextualized by previous interviews, to investigate common measures of adoption and their implications for CA's promotion, monitoring and evaluation. These farmers are unusually informative, because they are unsubsidized but have the capacity, incentive and willingness to adapt to climate change. We find that they are adopting CA autonomously, but that their implementation is highly variable and their interpretation of farming practice differs from that of local experts. Single proxies, binary adoption variables and broad farmer self-assessments suggest that between 40 and 80% of farmers have adopted CA. However, when evaluated across the three CA principles using UN-defined adoption thresholds, the comprehensive adoption rate is only 14%. Farmers' definition of "conservation" differs substantially from that of the local experts most likely to be asked to contribute adoption estimates to global monitoring efforts, creating the potential for miscommunication. There is therefore substantial cause for concern in how CA is currently promoted, monitored and evaluated. Inaccurate adoption estimates jeopardize CA's potential as a climate change adaptation strategy, creating illusory progress that may disincentivize further substantive efforts towards agricultural sustainability and climate resilience.