
First-night question volume as a signal of welcome-message quality
A measurement protocol for first-night guest questions, using reason codes and message timing to test whether the arrival note answers the questions guests actually ask.
Question volume is only meaningful when each contact is coded by reason and compared with the message version the guest received.
Methodology
- The unit of analysis is one guest stay, and the observation window runs from the scheduled check-in time to the first eight hours after arrival. A contact enters the population when the guest initiates a message, call, or platform request during that window. Coordinator-initiated check-ins are excluded because they measure the team's behavior rather than the guest's need. Stays with no contact remain in the denominator so the rate is not inflated by selection.
- A question is coded by the information the guest was trying to obtain, not by the wording of the message. The initial code set covers entry and access, Wi-Fi and devices, climate and hot water, amenities and supplies, parking and transport, and checkout or house rules. Codes are assigned from the guest's first sentence and can be revised only with a written note. A contact that contains several questions is counted once under the reason that triggered it, with the other topics recorded as secondary.
- Each stay is linked to the version of the welcome message that was sent and the channel that delivered it. A message sent on arrival morning is distinguished from one sent several days earlier, because timing changes whether the guest sees it near the moment of use. When the property has both a platform message and a printed card, the reviewer records whether the two sources agreed. Conflicting sources are treated as a separate signal rather than a guest error.
- The primary measure is the share of stays with at least one first-night contact, reported with the eligible stay count. Secondary measures include contacts per stay, the share of contacts whose reason was already covered in the message, and the time from arrival to the first contact. A separate count records contacts that required escalation beyond the coordinator. Percentages appear beside absolute counts, and no rate is reported for a reason code with fewer than five cases.
- Sampling uses consecutive stays in the window when volume is low and a fixed interval sample when volume exceeds one hundred stays. The frame is frozen before messages are read, and stays are not dropped because the guest seemed difficult or because the message was unusually long. Reviewers include at least one stay from each property type and at least one after-hours arrival. The stopping point is the end of the observation window for the last stay in the frame.
- Two reviewers code twenty contacts independently before comparing reason codes and message coverage. Disagreements are resolved by a team lead who records the definition that settles the case. Reliability is reported as the share of matching codes, with the ambiguous cases listed separately. The review does not ask the guest to justify the question, because the record is the contact itself.
- The report states the number of stays, the properties covered, the message versions in use, and the share of arrivals outside normal staffed hours. It distinguishes a local result from any claim about guest behavior in the wider market. The final section names one message block to revise, the owner of the change, and the date of the next observation window. No individual guest is identified in the published result.
- Properties are grouped by arrival complexity, because a building with a lobby, an elevator, and a parking garage produces different questions than a standalone cottage. The grouping is defined from the property brief before contacts are read, and a property that changes its access setup during the window is flagged. Reviewers do not compare a complex arrival directly with a simple one when reporting a rate. The grouping keeps the comparison honest without excusing a genuinely confusing message.
- The protocol records whether the guest opened the message and when, using the delivery data available in the platform. An unopened message is separated from an opened message that failed to answer the question, because the repairs differ. A guest who never opened the note may need a different delivery time, while a guest who read it may need clearer wording. The review does not treat an unopened message as proof that the content was adequate. The window closes before the report is circulated so late corrections cannot silently rewrite the record.
Findings
- Most first-night contacts trace to a small number of topics, and the exact ranking varies by property more than by season. Entry and access usually leads, followed by Wi-Fi and climate controls, but a property with a complicated parking arrangement can reverse that order. The ranking matters because it tells the team which paragraph to repair first. A general claim that guests ask too many questions does not support a specific change.
- Questions already covered in the message are the most useful signal, because they show a gap between what was written and what the guest could find. A long message can still fail when the relevant line sits below three screens of house rules. A short message can fail when the entry step assumes knowledge the guest does not have. Coverage is therefore tested by whether the guest acted on the information, not by whether the words exist.
- Message timing has a visible effect on the share of stays with a first-night contact. Messages sent several days before arrival are often buried under platform reminders, while messages sent on arrival morning are closer to the moment of use. The effect is not uniform, because some guests read everything in advance and others read nothing until the door. The protocol reports the timing groups separately rather than pooling them.
- Conflicting sources produce more contacts than missing details, because the guest cannot tell which version is current. A printed card with an old door code or a house guide with a removed amenity sends the guest back to the team even when the welcome message is correct. Reviewers record the conflict as its own reason code so the repair targets the source rather than the wording. This finding is consistent across property types in the reviewed cases.
- Contacts that escalate beyond the coordinator tend to involve access, safety, or a promise the guest believes was made. Those cases are expensive because they interrupt the duty queue and often involve the owner. A message that states the contact path and its hours reduces the chance that a routine question becomes an escalation. The protocol treats escalation as a separate outcome rather than folding it into question volume.
- The most effective repair is usually a rewrite of one block supported by a single source of truth. Teams that shortened the entry section and linked the rest of the guide saw contacts shift toward genuine questions rather than repeat requests. The local sample cannot establish causation, and the result should be retested after the next property change. A second window is required before the revision is treated as durable.
- Stays with a pre-arrival call from the coordinator show fewer first-night contacts, but the difference is not uniform across properties. The call may answer a question that the message would have answered later, which makes the message look better than it is. The protocol records coordinator-initiated contact separately so the effect can be seen. A team should not use an extra call to compensate for a weak written note.
- Guests who arrive after midnight produce a distinct pattern, with access and quiet-hours questions dominating and general amenity questions nearly absent. The small number of cases makes the pattern provisional, but it is consistent enough to justify a separate after-hours message. Reviewers report the after-hours group with its own denominator rather than merging it into the overall rate. That separation prevents a night-arrival problem from disappearing inside a daytime average.
- The protocol does not attempt to score guest satisfaction or review sentiment, because those outcomes are shaped by many factors outside the message. It asks a narrower question about whether the arrival note answered the questions guests actually brought to the team. A low contact rate is not automatically good if the guest simply gave up or asked a neighbor. The report pairs the rate with the reason codes so a quiet inbox is not mistaken for a clear message.
Research standards
Every review freezes its observation window before findings are calculated. Reviewers retain a row-level evidence ledger with the original artifact reference, timestamp, responsible role, status at cutoff, correction history, denominator, exclusions, and unresolved cases. Ambiguous records are coded independently before reconciliation, and both readings remain visible until an accountable adjudicator applies a documented rule. Missing information is treated as unknown rather than silently classified as success or failure. Sensitivity checks remove major record classes in turn so a pattern that depends on one owner, channel, event type, or reporting cycle is disclosed instead of presented as stable. Elapsed time is not translated into labor, an association is not described as causation, and local observations are not converted into external benchmarks.
Interpretation begins with counterexamples and the cases still open at the cutoff. A formally complete record may lack usable evidence, while an authorized alternative path may achieve the intended outcome without matching the expected workflow. The review therefore states the strongest alternative explanation, identifies the additional artifact that could distinguish it from the preferred reading, and records what would reverse or retire the resulting operating rule. Comparisons use the same field definitions and equivalent work classes; a material change in coverage, tooling, policy, or sample composition creates a new baseline rather than a trend. Follow-up records name the decision taken, responsible owner, anticipated signal, review date, exception path, and evidence required before a stronger intervention can be justified.
Governance checks keep only information necessary for the research question. Sensitive content is replaced with a controlled reference whenever an authorized reviewer can reproduce the classification without holding a copy. Access boundaries, retention expectations, correction ownership, disputed-interpretation routes, duplicate handling, spanning-record rules, and cutoff treatment are documented before release. Absolute counts appear beside percentages, small samples avoid false precision, and every external statement links to a listed source without implying that the publication measured the local record set. The proposed action must be proportionate, reversible, and reviewable. If the available evidence cannot support it, the gap becomes an explicit evidence request rather than an assumption.
This research note supports the STR Virtual Assistant topical map for vacation-rental guest communication, property readiness, issue routing, and evidence-based operational follow-through.
Sources
- NIST Cybersecurity FrameworkNISTChecked 2026-11-07
- NIST Privacy FrameworkNISTChecked 2026-11-07
- CISA Secure Our WorldCISAChecked 2026-11-07
- FTC Data SecurityFTCChecked 2026-11-07
- SBA Business GuideSBAChecked 2026-11-07
- DOL RecordkeepingDOLChecked 2026-11-07
- IRS RecordkeepingIRSChecked 2026-11-07
- GAO Assessing Data ReliabilityGAOChecked 2026-11-07
- NIST SP 800-53 Rev. 5NISTChecked 2026-11-07
- BLS Secretaries and Administrative AssistantsBLSChecked 2026-11-07
- HUD Assistance Animals GuidanceHUDChecked 2026-11-07