趣虹

Baidao helps Quhong with data analysis

Customer background

Hong Kong Quhong Technology Co., Ltd. was founded in 2014 by three young people with dreams, who had worked at IBM, Tencent, and Gionee. Quhong Mobile is guided by market demand and focuses on developing first-class and user-friendly applications. Quhong’s products cover the design and development of mobile applications, application redevelopment, Android application development, etc. In 2015, it had more than 15,000,000 application users. Quhong Mobile, leveraging its own traffic and the advantages of sufficient development partners, shifted to the Indian mobile advertising network. Quhong Mobile is committed to serving global advertisers and developers, maximizing user acquisition and traffic monetization. With the concept of putting customers first and employees at the core, it helps customers overcome all difficulties and obstacles on the mobile marketing path and provides a one-stop platform for employees’ lives and work.

industry

Social networking industry

Business requirements and challenges

With the growth of Quhong Technology’s business, Quhong Technology has acquired a large number of users. However, it has also encountered bottlenecks simultaneously: After the number of their users reaches a certain level, the growth becomes extremely slow, and they don’t know what methods can be used to boost it; they also need to optimize and upgrade some functions based on users’ preferences for the functions. Currently, they are still using the simplest method of questionnaires, and then manually sorting and analyzing the questionnaires; when preparing to launch some new functions, the display style of the launch notification messages cannot meet the preferences of all people, resulting in a relatively low click – through rate of the messages. They hope that the messages can be personalized and recommended according to users.

With the continuous growth of business, the management of the data analysis system also becomes extremely laborious. While the amount of data is increasing, expanding the self – built big data cluster is quite troublesome, and it also requires a large amount of human resources input to ensure the stable operation of the data analysis platform.

In terms of data storage, there is no unified storage method. Currently, the data is collected piecemeal from various places and then uploaded to the data analysis platform for data analysis. The entire process is rather lengthy, which is not conducive to the timely processing and analysis of data.

solution

In the project, Baidao, a Google partner, helped the IT team of Quhong conduct a detailed analysis of the cost investment planning and optimization plan for the cloud – based business of Google Cloud Technology. At the same time, it provided excellent answers and business solutions to various problems and requirements. During the transformation process, some Google Cloud products were used to assist Quhong in designing the architecture for data analysis, improving the old architecture, enabling customers to lean towards automated infrastructure management in data analysis, focus on data processing, and maintain the stable operation of the system. Its professional and meticulous service support left a deep impression on customers.

The architecture design diagram is as follows:

Dataproc optimizes the processing efficiency of data analysis

Dataproc is a managed big data processing service on the Google Cloud platform. It can provide users with a fast, flexible, and scalable Apache Hadoop and Apache Spark environment, as well as other related big data development tools and services. Dataproc has the powerful infrastructure and automated management capabilities of Google Cloud, enabling the rapid creation and management of clusters with hundreds or even thousands of nodes.

Dataproc also provides flexible scalability options to increase or decrease the scale of the cluster according to the requirements of the workload, so as to achieve high throughput and low latency when processing large-scale data. Dataproc also has automated cluster management functions, including auto-scaling, auto-fault tolerance, and auto-monitoring, etc., enabling users to focus on data processing tasks without having to worry about the management and maintenance of the underlying infrastructure.

In the past, Quhong used a traditional self – built Hadoop cluster. Although it was built on cloud virtual machines in the cloud, it was very labor – intensive in terms of operation and maintenance difficulty and cluster expansion and contraction. Using Dataproc can reduce the difficulty of operation and maintenance. Since it is a managed service, customers only need to manage the workloads they run, without having to worry about the underlying infrastructure. Moreover, Dataproc can adjust the size of the cluster according to the real – time workload situation to ensure the availability of resources.

Build an enterprise – level data warehouse with BigQuery

BigQuery是谷歌云的一种强大而全面的云原BigQuery is a powerful and comprehensive cloud – native data warehouse and analytics engine of Google Cloud. It is a serverless data analytics solution, eliminating the need to manage infrastructure and configurations. Users can simply focus on data and analysis. At the same time, BigQuery has the ability to scale elastically, automatically adapting to large – scale datasets and concurrent queries, ensuring high – performance and low – latency query results. BigQuery utilizes distributed computing and column – based storage technologies, endowing it with excellent query performance. It can handle billions or even trillions of rows of data and return query results within seconds or even sub – seconds, accelerating the data analysis and decision – making process. Moreover, BigQuery can be easily and closely integrated with other Google Cloud services and tools, such as Google Cloud Storage, Dataproc, Dataflow, etc. Data can be effortlessly imported into BigQuery for analysis, and query results can be exported to other services for further processing. 

In the past, the storage of data used for analysis by Quhong was extremely messy. The data was stored in various ways, such as in local machine rooms and MySQL databases. This led to the need to design multiple data pipelines for data analysis, making the entire data analysis platform particularly bloated. After using BigQuery, Quhong can have a unified data warehouse, and the design of data processing pipelines will be more streamlined. Moreover, with the powerful query performance of BigQuery, data can be pre – processed quickly in advance, shortening the data cleaning cycle. 

Some analytical data do not require complex processing and can be directly applied to data reports after being processed by BigQuery. Combining BigQuery with Looker Studio can quickly generate reports and obtain some preliminary information in advance.

Use the elastic scaling capabilities of the cloud to build a powerful and stable display platform

By leveraging the elastic scaling capabilities of Google Cloud, it can provide a highly flexible resource management and scalability for Quhong’s data display platform, enabling it to adapt to the constantly changing display scale and traffic demands. The display platform can automatically adjust the scale and capacity of resources according to actual needs. This means that during the peak period of viewing display reports, the platform can quickly expand to ensure stable performance and user experience; while during the off – peak period, the platform can automatically reduce resources to save costs. This flexibility not only provides efficient resource utilization but also ensures the stability and reliability of the data display platform.

Google Cloud also provides advanced security and reliability guarantees. Data is backed up and redundantly stored in the cloud to prevent data loss and corruption. At the same time, Google Cloud’s security measures and access control mechanisms ensure that the content of the display platform and user data are fully protected.

Use the product •

• Compute Engine
• Dataproc
• Load Balancing
• Cloud CDN
• Google Cloud Storage
• Firebase
• BigQuery
• Dataflow
• Pub/Sub

Customer earnings

Note: 【The following statistical data are all from the statistical results of the customer background】

• By adding an additional data tagging method of Pub/Sub, some data can reach the data warehouse in real time without waiting for the T+2 mechanism of Firebase, reducing the data collection time by 60%.
• After using BigQuery, Quhong has a unified data warehouse, making data management more standardized.
• Improve the network connection stability by 20% through Google’s high-quality private network.
• Scale on demand according to the business development speed, achieve zero latency in hardware procurement, configuration and maintenance, and also save 30% of the labor cost.
• Using Dataproc, compared with the Hadoop cluster previously built by the customer themselves, the customer’s maintenance time has been reduced by 70%. On the operational side, a large amount of data analysis time has also been saved. This enables the company’s strategies and marketing activities to be implemented more quickly.