CBA-DMP · Management
Specimen paper
Twelve examination items for the CBA Certified Digital Marketing Professional, with the answer and a rationale for every option.
Examiner’s note
This specimen is twelve items set out in the proportions of the live paper: all six domains, from campaign economics and multilingual content through search, social, creator and messaging channels, and email and conversion, to measurement and testing, with four foundational items, six standard and two demanding. Every item is set in a working marketing operation, any text in another language comes with a literal English back-translation, and the arithmetic, where it appears, is expected to be exact. The commonest way of losing marks on this certification is not a gap in channel knowledge but measuring with the wrong quantity: reading revenue as profit, a platform's reported conversions as conversions the spending caused, an average return as the return on the next unit of spend, or a count as a rate. Several options here are written to reward precisely that. Establish what each figure measures before you compare it with anything.
Items
12
Domains
6
Questions in the examination
60
- 01Campaign Planning, Economics and BudgetsFoundational
Lamsa Scents, an online perfume retailer in Jeddah, earns a gross margin of 40 per cent on net revenue; after payment, delivery and returns handling, its contribution margin is 25 per cent of net revenue. The revenue below is net of VAT and returns, on the same basis as the margin. Campaign A | spend SAR 8,000 | revenue SAR 40,000 Campaign B | spend SAR 12,000 | revenue SAR 36,000 Campaign C | spend SAR 6,000 | revenue SAR 12,000 Which campaigns lose money once contribution is set against spend?
- A
None of them, as each campaign brings in more revenue than it spends.
This treats any ROAS above one as profit, setting spend against revenue rather than against the contribution that revenue leaves. At a 25 per cent contribution margin, each SAR 4 of revenue leaves SAR 1 to pay for the advertising. It would be right only if the business kept every riyal of revenue, which no seller of goods does.
- B
C alone, as only its ROAS of 2.0 falls below the break-even of 2.5.
This works break-even from the 40 per cent gross margin, before payment, delivery and returns handling have taken their share, and so sets the bar at 2.5 instead of 4.0. Campaign B passes that test while losing SAR 3,000. It would be right if gross margin were the whole cost of a sale, with nothing to pay to deliver it or take it back.
- C
B and C, as their ROAS of 3.0 and 2.0 fall below the break-even of 4.0.
Correct: break-even ROAS is one divided by the 25 per cent contribution margin, or 4.0, and B and C each lose SAR 3,000 against it.
- D
All three, as the blended ROAS of about 3.4 falls below the break-even of 4.0.
This judges the account as a whole and then condemns every campaign in it. The blend is pulled down by B and C, while A returns SAR 10,000 of contribution on SAR 8,000 of spend and earns SAR 2,000. It would be the right reading of a question about whether the account as a whole loses money, which it does, by SAR 4,000.
Why that is the answer
Return on ad spend divides revenue by spend, but the advertising is paid for out of contribution, not revenue, so the break-even line is one divided by the contribution margin rate: at 25 per cent, every SAR 4 of revenue leaves SAR 1 to cover the media. Campaign B's ROAS of 3.0 looks healthy on a dashboard and loses SAR 3,000, because SAR 36,000 of revenue leaves SAR 9,000 of contribution against SAR 12,000 of spend. Two checks come before the comparison: that the margin is taken after payment, delivery and returns handling, and that the revenue is on the same basis as the margin, net of VAT and returns. Then judge each campaign against the line, not the account's average.
- 02Campaign Planning, Economics and BudgetsStandard
Rasif Lane, a fashion retailer in Dubai, spends AED 30,000 a month on each of two channels and has AED 10,000 more to place. Its analyst has estimated the contribution each successive AED 10,000 of spend returns, after cost of goods, payment, delivery and returns but before the media cost itself. Treat the contribution within each tranche as spread evenly across it. Search | first AED 10,000: AED 20,000 | second: AED 15,000 | third: AED 11,000 | fourth: AED 8,000 Social | first AED 10,000: AED 13,000 | second: AED 12,500 | third: AED 12,000 | fourth: AED 11,500 Where should the extra AED 10,000 go?
- A
Social, whose fourth AED 10,000 returns AED 11,500 and so adds AED 1,500.
Correct: the fourth tranche in Social returns AED 11,500 of contribution for AED 10,000 of spend, while the same tranche in Search returns AED 8,000 and loses AED 2,000.
- B
Search, whose average return of about 1.53 is the best of the two channels.
This sends the next tranche where the average is highest, but the average is carried by Search's first AED 10,000, which returned AED 20,000 and will not be earned again. The next AED 10,000 in Search returns AED 8,000, a loss of AED 2,000. Average and marginal returns would point the same way only if returns did not diminish with spend.
- C
Split it evenly, so neither channel is pushed far into diminishing returns.
This treats balance as a virtue in itself. Half of the tranche would go into Search's fourth tier, where each AED 1 returns AED 0.80, so the split returns AED 4,000 plus AED 5,750, or AED 9,750, against AED 11,500 for placing all of it in Social. An even split would be right only if the two channels' next tranches returned the same.
- D
Hold it back, since adding spend to either channel lowers its average return.
This manages to the average, which falls with almost any extra spend once returns diminish, and so would refuse spending that pays. Social's fourth tranche lowers its average from 1.25 to 1.225 and still adds AED 1,500 of contribution. Holding back would be right only if no channel's next tranche returned more than it cost.
Why that is the answer
Budget should follow the return on the next unit of spend, not the return on the spend already placed. Search's average of about 1.53 is real but historical: it is carried by a first tranche that returned AED 20,000, and its fourth tranche returns AED 8,000, which loses AED 2,000 once the media is paid for. Social's average is lower, but its fourth tranche returns AED 11,500 and adds AED 1,500. Read spend-tier data at the margin: place each increment where the next tranche earns most, and stop where no channel's next tranche earns more than it costs.
- 03Multilingual Content, Creative and ClaimsFoundational
Lubban Electronics in Muscat is clearing a bilingual copy deck for a sale banner. Legal has approved these offer terms: 25 per cent off all laptops, 1 to 15 May. Each Arabic line is shown by its literal English back-translation. Row 1 | headline | English | limit 30 | count 22 | text: Big savings on laptops Row 2 | headline | Arabic | limit 30 | count 26 | back-translation: Savings await you on laptops Row 3 | offer line | English | limit 40 | count 32 | text: 25% off all laptops, 1 to 15 May Row 4 | offer line | Arabic | limit 40 | count 35 | back-translation: Discount up to 25% on laptops, 1 to 15 May What should the reviewer do?
- A
Return row 2, as a headline should render the English one word for word.
This holds that literal translation is the safe course, when a headline is exactly where adaptation is allowed: 'savings await you' carries the same selling message as the English in a form natural to Arabic readers, and fits its limit. Row 2 would need returning only if it changed a fact, an offer term or a mandatory inclusion, or ran over its 30 characters.
- B
Return rows 2 and 4 for length, as Arabic runs longer than its English source.
This assumes Arabic always overruns its English source, when the deck supplies the counts: row 2 runs to 26 characters against a limit of 30 and row 4 to 35 against 40, so both fit. Length is checked line by line against each line's own limit, and would be a reason to return a line only if its count exceeded that limit.
- C
Approve the deck, since transcreation may adapt the wording of the offer.
This extends the licence of transcreation to the offer, when offer terms are regulated wording that must match the approved terms in every language. 'Up to 25 per cent' promises less than 25 per cent off all laptops, so the two audiences would be offered different deals. Approval would be right only if both offer lines stated the approved terms.
- D
Return row 4, as 'up to 25 per cent' does not state the approved offer.
Correct: the Arabic offer line turns 25 per cent off all laptops into a discount of up to 25 per cent and drops 'all', so it no longer states the approved terms, and only the approved terms may run, in every language.
Why that is the answer
A bilingual deck is cleared line by line against the brief, and different lines carry different licences. A headline may be transcreated, so the Arabic can sell the same idea in its own idiom provided it fits the placement; an offer line is regulated wording, and must state the approved terms exactly in each language. Row 4 turns 25 per cent off all laptops into up to 25 per cent off, which offers Arabic readers a different and lesser deal from the one shown in English. Check the offer terms in every version against the approved wording, and check each line's character count against its own limit, rather than assuming either from the other language.
- 04Multilingual Content, Creative and ClaimsStandard
Rimth Coffee, a roaster in Manama, is clearing copy for a gift box of its new blend. Its claims policy requires every claim to be specific, to cover only the part of the product the evidence covers, and to be supported by evidence held before publication. The evidence held is a supplier certificate dated March 2026 stating that the outer carton contains 60 per cent recycled fibre by weight. The coffee pouches inside are made from new plastic, and no comparative testing has been done. Which line may be published?
- A
Eco-friendly packaging, since the carton is made with recycled fibre.
This lets one recycled component support a general environmental claim. 'Eco-friendly' is vague, and it describes a pack whose pouches are new plastic. It could be right only if evidence covered the whole pack and the policy accepted general claims, and this policy does not.
- B
Outer carton made with 60 per cent recycled fibre, as the certificate states.
Correct: the claim is specific, names the component the certificate covers and repeats its figure, so every word is supported by evidence held.
- C
Packaging made with 60 per cent recycled material, as the certificate states.
This is the closest wrong answer, because the figure is right and the wording sounds precise. 'Packaging' takes in the pouches, which are new plastic, so a figure proved for the carton is applied to the whole pack. It would be right only if the certificate covered every component and the pack as a whole met that figure.
- D
The greenest coffee packaging in Bahrain, as other roasters claim the same.
This is a comparative claim with no comparative evidence behind it, defended by what others say. That other roasters make the claim is not evidence for this one. It would be publishable only with comparative testing against the packaging it is compared with, held before publication.
Why that is the answer
A claim is only as wide as its evidence. The certificate proves one fact about one component, 60 per cent recycled fibre in the outer carton, so the publishable claim names that component and that figure and stops there. Widening it to 'packaging' carries the figure across pouches made of new plastic; generalising it to 'eco-friendly' asserts something about the whole product that nothing held supports; and a comparison needs comparative testing, which no amount of competitors' copy replaces. Before clearing a line, set each word of the claim against the evidence and strike whatever the evidence does not cover.
- 05Organic and Paid SearchStandard
Mihwar Training, a professional training institute in Amman, wants 150 enquiry leads a month from paid search at a target cost of JOD 50 a lead, and has set its monthly budget at JOD 7,500. Its landing page converts 4 per cent of clicks into leads. The planning tool quotes JOD 3.00 a click for the course keywords. What should the marketing manager do?
- A
Proceed, as JOD 7,500 at the JOD 50 target buys exactly 150 leads.
This divides the budget by the target and treats the result as a forecast. A budget set at the target cost per lead always buys the goal on paper, because the conversion rate cancels out of the sum; what decides the matter is whether a click can be bought at the affordable price. It would be right only if the quoted click cost were JOD 2.00 or less.
- B
Revise the plan, as the affordable CPC is JOD 2.00 against a quote of JOD 3.00.
Correct: the affordable CPC is the JOD 50 target times the 4 per cent conversion rate, or JOD 2.00; at the quoted JOD 3.00 each lead costs JOD 75, so the budget buys 100 leads, not 150.
- C
Proceed, as a JOD 3.00 click costs far less than the JOD 50 allowed for a lead.
This compares the price of a click with the price of a lead, which are different units. Only 4 clicks in 100 become a lead, so 25 clicks are bought for each one, and at JOD 3.00 that is JOD 75. The comparison would be valid only if every click became a lead.
- D
Raise the bids, as a JOD 3.00 quote will not win the 3,750 clicks the goal needs.
This treats the shortfall as a lack of clicks, when it is a matter of price: a click at JOD 3.00 already costs more than the JOD 2.00 the target allows, and a higher bid raises that price further, so every lead would cost more than JOD 75. A higher bid would be the answer only if the quoted CPC were affordable and the campaign were losing auctions it could afford to win.
Why that is the answer
Work backwards from the lead to the click. If a lead may cost JOD 50 and 4 clicks in 100 become leads, a click may cost no more than JOD 50 times 0.04, which is JOD 2.00. The quoted JOD 3.00 is half as much again, so each lead costs JOD 75 and the JOD 7,500 budget yields 100 leads. Notice why the budget alone could never reveal this: 150 leads at JOD 50 is JOD 7,500 whatever the conversion rate, so the affordable CPC set against the quote is the test that decides whether the goal is reachable. Where it is not, the levers are a higher conversion rate, a different keyword set or a revised goal, not a higher bid.
- 06Organic and Paid SearchFoundational
Sahat Kitchen, an appliance retailer in Abu Dhabi, publishes a guide that ranks for a question about descaling kettles and takes almost all its organic clicks from that query. Organic clicks to the guide fell by 40 per cent year on year in March. Its search console shows, for that query: March 2025 | average position 2.4 | impressions 50,000 | clicks 5,000 | click-through 10.0% March 2026 | average position 2.1 | impressions 60,000 | clicks 3,000 | click-through 5.0% What best explains the fall?
- A
Lost rankings, since a 40 per cent fall in clicks marks a demoted page.
This reads every traffic fall as a demotion or penalty, when the guide's average position improved from 2.4 to 2.1. A loss of rankings shows first in position, and then in impressions as the page drops out of view. It would be right if position had worsened while impressions fell.
- B
Lower demand, since fewer people searched for the question this March than last.
This is the right first question, since seasonal demand is checked year on year, and the data answers it: with the guide near the top of the results in both years, impressions for the query rose from 50,000 to 60,000, so more people searched, not fewer. It would be right if impressions had fallen in step with clicks while position held.
- C
A slow page, since visitors who click are leaving before the guide loads.
A slow page loses visitors after they click, but a click on the results page is counted when it is made, so page speed cannot lower the click-through the console reports. It would be a fair diagnosis if clicks held while sessions, enquiries or sales from the guide fell.
- D
Results-page features answering the query, so fewer searchers need to click.
Correct: the guide ranks higher and is shown more often for the query, yet the share of searchers who click has halved, which is the indirect signature of the results page answering the question itself.
Why that is the answer
Read a traffic fall through its three components before naming a cause: position says whether the page lost ground, impressions for a page that holds its place near the top say whether demand changed, and click-through says whether searchers who saw the page chose it. Here position improved and impressions rose by a fifth, so neither rankings nor demand explains the loss; the whole of it sits in click-through, which halved from 10 per cent to 5 per cent. For a question with a short factual answer, that pattern points to the results page answering the question itself, so searchers see the guide and have no need to visit it. The response is not to repair a ranking that is not broken, but to judge what the guide still offers a searcher who has already read the short answer.
- 07Social, Creator and Messaging ChannelsStandard
Bawabat Fit, a fitness app in Doha, will engage one creator for a launch aimed at residents of Qatar. Its criteria are: at least 60 per cent of the audience in Qatar; an engagement rate of at least 5 per cent, taken as engagements (likes, comments, shares and saves) divided by reach; and no sudden jump in followers in the past six months. Creator A | followers 400,000 | reach 40,000 | audience in Qatar 30% | engagements 1,600 | follower growth steady Creator B | followers 120,000 | reach 20,000 | audience in Qatar 65% | engagements 1,800 | gained 50,000 followers in one week, two months ago Creator C | followers 90,000 | reach 30,000 | audience in Qatar 70% | engagements 2,100 | follower growth steady Creator D | followers 60,000 | reach 25,000 | audience in Qatar 80% | engagements 1,000 | follower growth steady Which creator meets all three criteria?
- A
Creator A, whose 400,000 followers give the launch the widest audience.
This takes follower count as a measure of influence. Only 30 per cent of A's audience is in Qatar, and 1,600 engagements on a reach of 40,000 is 4 per cent, so A fails two of the three criteria. It would be the right choice only if the brief rewarded audience size alone, with no market or engagement test.
- B
Creator B, whose engagement rate of 9 per cent is the highest of the four.
B has the highest rate, 1,800 engagements on 20,000 reached, and passes the market test, which makes this the strongest wrong answer. A gain of 50,000 followers in a single week two months ago falls inside the six months the third criterion covers, and is the authenticity signal it exists to catch, since bought or prompted followers can inflate the counts. It would be the right choice only if the criteria allowed a jump with a verified cause, such as a broadcast appearance, and this one had one.
- C
Creator C, whose 70 per cent in Qatar comes with 7 per cent engagement.
Correct: C has 70 per cent of its audience in Qatar, engages 2,100 of the 30,000 it reaches, a rate of 7 per cent, and has grown steadily.
- D
Creator D, whose 80 per cent in Qatar is the closest match to the market.
D has the strongest audience match, but 1,000 engagements on 25,000 reached is 4 per cent, below the 5 per cent required. This answer stops at the first criterion it can check. It would be right if the brief asked for audience in Qatar alone, with no engagement threshold.
Why that is the answer
A shortlist is judged against every criterion in turn, and each figure must be worked on the stated base. Engagement rate here is engagements divided by reach, so A and D both come out at 4 per cent and fail, while B at 9 per cent and C at 7 per cent pass; divided by followers instead, every creator on the list would fail. Of the two that clear the rate, B gained 50,000 followers in one week two months ago, inside the six-month window and the pattern that bought or prompted followers leave, so the third criterion excludes it. Follower count settles none of this: C has fewer than a quarter of A's followers and is the only creator who fits the brief.
- 08Social, Creator and Messaging ChannelsFoundational
Zahrat Care, a skincare brand in Sharjah, received this public comment, written in Gulf dialect, under a post about its new face cream: طلبي الثاني ما وصل للحين، ومن يوم استخدمت الكريم من الطلب الأول طلع لي حبوب وحكة في وجهي. وين رقم التتبع؟ Back-translation: My second order has not arrived until now, and from the day I used the cream from the first order pimples and itching came out on my face. Where is the tracking number? Its comment policy says: reply publicly to questions about products, prices and delivery; move to a private channel any matter that needs an order number or personal details to resolve; escalate to the quality team the same day any report of a reaction, injury or safety concern, and post a short public acknowledgement; never hide or delete a comment unless it is abusive or spam. Where more than one step applies, escalation comes first. What should the community manager do first?
- A
Escalate it to the quality team today, and acknowledge it in a public reply.
Correct: the comment reports a skin reaction, which the policy sends to the quality team the same day with a short public acknowledgement; the missing order can then be taken to a private channel.
- B
Escalate it to the quality team today, and hide the comment until resolved.
This escalates correctly and then removes the comment, which the policy permits only for abuse or spam, and a hidden safety report tends to be noticed and to do more damage than the comment itself. It would be right only if the comment were abusive or spam, which a report of a reaction is not.
- C
Escalate it to the quality team today, and handle it in private messages only.
This escalates correctly but leaves the public thread unanswered, when the policy requires a short public acknowledgement of a reaction report, so that other customers can see it has been taken seriously. Taking the missing order to a private channel is right as a later step, alongside that acknowledgement rather than instead of it. It would be right if the comment had raised only the missing order.
- D
Reply publicly with the tracking details, then pass the reaction to quality.
This deals with the delivery question first, and in public, when the policy puts escalation first and sends anything that needs an order number to a private channel, which is where a tracking number belongs. It would be right only for a comment that asked a general delivery question needing no order details, such as how long delivery takes.
Why that is the answer
A comment policy works by matching the comment to the step it names, and this one also says which step comes first when more than one applies. The comment raises two matters: an order that has not arrived, which needs an order number and so belongs in a private channel, and a reaction to the cream, which goes to the quality team the same day with a short public reply that shows other customers it has been taken seriously. Escalation comes first, but escalating is not the whole of the step: the public acknowledgement is part of it, hiding the comment is ruled out by the policy, and answering the delivery question in public puts order details where they do not belong. Read the whole comment for every matter it raises, then apply every part of the step it triggers.
- 09CRM, Email and Conversion OptimisationStandard
Sanad Cover, a car insurance broker in Kuwait City, tracks its online quote journey for one month: Stage 1 | quote page visits | 20,000 users Stage 2 | started the quote | 10,000 users Stage 3 | completed vehicle details | 4,000 users Stage 4 | viewed the price | 3,600 users Stage 5 | submitted contact details | 2,700 users The team will test one change first, at the step with the lowest stage conversion rate, taken as each stage's users divided by the users at the stage before. Which change should it test?
- A
Rework the quote page, which loses 10,000 visitors, more than any other step.
This reads largest loss as most users lost, and on that reading the quote page would come first, since 10,000 of 20,000 visitors leave without starting. The team defined the test by stage rate, where the quote page converts at 50 per cent and the vehicle-details step at 40. It would be right if the team had chosen to work on the step that loses most users.
- B
Shorten the contact form, which only 13.5 per cent of all visitors submit.
This divides by the top of the funnel instead of the stage before, which makes the last stage look worst whatever happens there. Measured on the users who viewed the price, 2,700 of 3,600, the contact form converts at 75 per cent. It would be right if the question asked which stage the fewest visitors reach, a measure of the whole journey rather than of one step.
- C
Buy more traffic for the quote page, so that more users enter the funnel.
This adds users to a journey without repairing it, so 60 in every 100 who start the quote would still be lost at the vehicle-details step, at the cost of the extra traffic. More traffic would be the right move only once each step converted as well as it reasonably could and the goal were more submissions in total.
- D
Shorten the vehicle-details step, which 4,000 of 10,000 starters complete.
Correct: 4,000 of the 10,000 who started the quote completed vehicle details, a stage rate of 40 per cent, lower than the 50, 90 and 75 per cent of the other steps.
Why that is the answer
A funnel is diagnosed step by step: each stage's users divided by the users who reached the stage before it, which isolates how well that one step does its job. Here the rates are 50, 40, 90 and 75 per cent, so the vehicle-details step, which loses six in ten of the people who began a quote, is the weakest step and the first place to test a change. Two other readings are tempting and answer different questions: dividing by the top of the funnel measures the journey to that point and always condemns the last step, and counting users lost favours the widest stage. Settle which definition of the largest loss applies before you calculate, because the same funnel gives different answers under each.
- 10CRM, Email and Conversion OptimisationDemanding
Wahat Pets, an online pet-supplies retailer in Riyadh, plans to upload contacts to an advertising platform on 1 October 2026 to show them a promotion. The rule it applies in this market: an upload for advertising needs prior, documented opt-in consent to marketing, given to Wahat Pets or naming it, and given separately from acceptance of terms; anyone who has opted out of any marketing, or complained, is suppressed from sends and uploads alike; and contact data may be used for 24 months from the latest consent or purchase, and not after. Contact 1 | source: own newsletter form, marketing box unticked until the person ticked it | consent recorded 12 May 2026 | status: active Contact 2 | source: own checkout, one box accepting the terms and marketing together | consent recorded 3 February 2026 | status: active Contact 3 | source: own newsletter form, marketing box unticked until the person ticked it | consent recorded 20 March 2025 | status: unsubscribed from email 2 August 2026 Contact 4 | source: list bought from a data broker, consent given to the broker for offers from selected partners, none of them named | consent recorded June 2026 | status: active Contact 5 | source: own event sign-up, marketing box unticked until the person ticked it | consent recorded 15 June 2024 | status: no purchase or renewed consent since Which contacts may be uploaded?
- A
Contacts 1 and 3, as an email unsubscribe does not reach uploads.
This assumes suppression governs sends but not uploads, when the stated rule suppresses anyone who has opted out of any marketing from both. Uploading Contact 3 would show advertising to a person who has said no to marketing. It would be right only under a rule that confined each opt-out to its own channel, and this one does not.
- B
Contact 1 only, the one valid, current consent with no opt-out.
Correct: Contact 1 gave separate, documented opt-in consent to Wahat Pets in May 2026 and has not opted out, while every other contact fails one part of the stated rule.
- C
Contacts 1, 2 and 5, each of whom gave consent the business recorded.
This counts any recorded consent as valid. Contact 2's box accepted the terms and marketing together, which the rule excludes, and Contact 5's consent dates from 15 June 2024, so its 24 months ran out on 15 June 2026. Both would be eligible only if consent had been given separately and within the retention period.
- D
Contacts 1 and 4, as the broker's list was collected with consent.
This accepts a list as compliant because it came with consent attached. Contact 4 consented, through the broker, to offers from partners it did not name, so the consent was neither given to Wahat Pets nor names it, as the rule requires. It would be eligible only if the consent recorded by the broker had named Wahat Pets as the business that would contact the person.
Why that is the answer
Eligibility is decided contact by contact against every part of the stated rule, and failing any one part removes the contact. Consent must be an opt-in the person chose, separate from accepting terms, which rules out Contact 2; it must be given to this business or name it, which consent to a broker's unnamed partners does not; it must still be within the retention period, which Contact 5's left in June 2026; and an opt-out from any marketing suppresses the person from uploads as well as sends, which removes Contact 3. That leaves Contact 1. Suppress the others rather than delete them, so that a later import cannot quietly restore them to the list.
- 11Measurement, Attribution and TestingDemanding
Nuqta Gadgets, an online electronics accessories retailer in Dammam, ran a month of retargeting ads with a randomised holdout: comparable recent site visitors were split at random into a group that could see the ads and a group that could not. Spend was SAR 24,000, and the ad platform reported 600 conversions. Exposed group | 40,000 visitors | 1,200 orders Holdout group | 10,000 visitors | 200 orders What was the cost per incremental conversion?
- A
SAR 40, from the 600 conversions the platform reports for the campaign.
This takes platform-reported conversions as incremental. The platform counts orders from people who saw or clicked an ad, including some who would have bought anyway, and here it claims 600 where the test shows 400. Its figure would be the right one only if none of the people it credits would have ordered without the ads.
- B
SAR 20, from all 1,200 orders placed by visitors in the exposed group.
This counts every order from the exposed group as caused by the ads, ignoring the 2 per cent who order with no ads at all, which the holdout measures. It would be right only if the holdout had placed no orders, showing that nobody in this audience buys unprompted.
- C
SAR 60, from 400 orders above the 800 the holdout rate predicts.
Correct: the holdout orders at 2 per cent, so the exposed 40,000 would have placed 800 orders anyway; the ads added 400, and SAR 24,000 divided by 400 is SAR 60.
- D
SAR 24, from 1,000 orders, the exposed group's less the holdout's.
This subtracts the holdout's orders as counted, without scaling them to the exposed group, which is four times its size. It compares 40,000 people with 10,000 and so overstates the effect two and a half times. It would be right only if the two groups were the same size.
Why that is the answer
Incremental conversions are the orders the ads caused: what the exposed group did, less what it would have done without the ads, and the holdout supplies the second figure as a rate. Here 200 orders from 10,000 held-out visitors is 2 per cent, so the 40,000 exposed visitors would have placed 800 orders unprompted; they placed 1,200, so the ads added 400, at SAR 24,000 divided by 400, or SAR 60 each. The platform's 600 conversions imply SAR 40, and the true cost is half as much again, because the platform credits orders that would have happened anyway. Scale the holdout to the exposed group before subtracting, and judge spend on the cost per incremental conversion, not the cost per reported one.
- 12Measurement, Attribution and TestingStandard
Diwan Living, an online furniture retailer in Beirut, is testing a new product page that changes the photo layout, the delivery promise and the button colour at once. The test was planned to run for 14 days with 14,000 visitors in each variant. After 4 days the readout shows: Control | visitors 4,000 | orders 120 | conversion 3.0% New page | visitors 4,000 | orders 156 | conversion 3.9% Stated confidence: 97 per cent The team acts on results that reach 95 per cent confidence at the planned sample size. The e-commerce manager wants to roll out the new page now and credit the delivery promise. What should the team do?
- A
Run to the planned 14 days, and credit any lift to the three changes together.
Correct: the test has run four of its fourteen days and under a third of its planned sample, and with three changes made at once any lift belongs to the package, not to any one of them.
- B
Roll out now and credit the page as a whole, as 97 per cent clears the 95 per cent bar.
This credits the lift to the right thing, the page as a whole, but stops the test at its first significant reading, before the planned sample the team's own rule requires. Confidence read repeatedly on a small, early sample crosses a threshold by chance far more often than its label suggests, and four days covers less than a full weekly cycle. A roll-out would be justified only once the planned sample and duration were complete and the result held.
- C
Extend the test to 21 days, then credit any lift to the delivery promise alone.
This adds time the decision rule does not call for and gets the attribution wrong. The test cannot separate the delivery promise from the photo layout or the button colour, since all three changed together, so even a longer result belongs to the package. Crediting one change would be right only after a test that varied it alone.
- D
Stop the test, as a 30 per cent lift in four days is most likely a novelty effect.
This turns a reason to keep testing into a reason to stop. A novelty effect is a real risk in an early lift, which is why the test runs its full length, but nothing in four days of data shows the lift is novelty. Stopping would be right if the test were found to be broken, such as uneven assignment between variants.
Why that is the answer
A test readout earns a decision only when the test has run as planned. This one has run 4 of its 14 days and gathered 4,000 of the 14,000 visitors planned for each variant, so its 97 per cent is a reading taken early and on a small sample, where lifts are unstable and an early lift may owe something to novelty. Even a confirmed result would belong to the new page as a whole, because three changes were made at once and nothing in the design can say which of them did the work. Let the test finish, judge the full result, and if the delivery promise matters on its own, test it on its own.
About these items
These twelve items are written to the specification of the live CBA-DMP paper, and none of them will appear on one. Every item in the bank is reviewed by a named subject-matter expert and audited for answer cueing domain by domain.
