Etsy has filed a patent for a method to detect and correct imbalances in online experiments. The method involves obtaining user information, generating alternative samplings of devices, and generating a threshold to detect imbalances. Once an imbalance is detected, a corrective action is implemented to mitigate it in the multivariate testing framework. GlobalData’s report on Etsy gives a 360-degree view of the company including its patenting strategy. Buy the report here.
Smarter leaders trust GlobalData
According to GlobalData’s company profile on Etsy, dynamic premium pricing was a key innovation area identified from patents. Etsy's grant share as of September 2023 was 52%. Grant share is based on the ratio of number of grants to total number of patents.
Imbalance detection in online experiments using multivariate testing framework
A recently filed patent (Publication Number: US20230273874A1) describes a method for detecting and mitigating imbalances in a multivariate testing framework. The method involves obtaining user information that indicates which devices are assigned to different versions of a service. Alternative samplings of devices are generated, and an expected set of devices for each version of the service is also generated. A threshold is then generated to detect imbalances using the alternative samplings and the expected sets of devices. The method detects imbalances by comparing the obtained user information with the generated threshold and implements corrective actions to mitigate the detected imbalances in the multivariate testing framework.
The user information used in the method can be obtained from a computer system that provides the different versions of the service to the assigned devices. The alternative samplings of devices assigned to each version of the service can be generated by performing permutations on values representing segments or variants of the user information. Additionally, the alternative samplings can be generated by sampling from a distribution that indicates the numerical range of device types assigned to each version of the service, where the assignment of device types is independent of the types of devices.
The expected sets of devices assigned to each version of the service are generated by determining the number of devices of a specific type that receive each version. These specific types can include devices operating in a specific region, with a specific language preference, on specific hardware, or with specific software.
To detect imbalances, the method generates one or more difference values between the obtained user information and the expected sets of devices. If the cumulative difference value exceeds a threshold difference value generated using the alternative samplings and the expected sets of devices, an imbalance is detected.
The method also includes implementing corrective actions to mitigate the detected imbalances. These actions can include generating notifications for testing devices and transmitting them to halt data processing processes on the testing device. Alternatively, a signal can be generated to assign additional devices to a specific version of the service to reduce the detected imbalance. The additional devices can be of a particular type represented by device or user features associated with the detected imbalance.
The patent also describes a non-transitory computer-readable medium that stores instructions for performing the method and a system comprising processors and machine-readable media that execute the instructions to perform the method.
To know more about GlobalData’s detailed insights on Etsy, buy the report here.
Data Insights
From
The gold standard of business intelligence.
Blending expert knowledge with cutting-edge technology, GlobalData’s unrivalled proprietary data will enable you to decode what’s happening in your market. You can make better informed decisions and gain a future-proof advantage over your competitors.
GlobalData, the leading provider of industry intelligence, provided the underlying data, research, and analysis used to produce this article.
GlobalData Patent Analytics tracks bibliographic data, legal events data, point in time patent ownerships, and backward and forward citations from global patenting offices. Textual analysis and official patent classifications are used to group patents into key thematic areas and link them to specific companies across the world’s largest industries.
