Baidao helps Fuliao with data analysis

Hangzhou Fuliao Technology Co., Ltd. was registered and established on July 19, 2013, with the Market Supervision and Administration Bureau of the High-tech Zone (Binjiang) in Hangzhou. The company’s business scope includes publishing domestic online advertisements on its own ipaychat.com website, etc., which is a product of Hangzhou Fuliao Technology Co., Ltd. Fuliao allows users to earn points by sending photos, etc. It is a social application where users can chat with others and exchange points for prizes. The company collects the browsing behaviors and content viewing data of each user, analyzes them to form user portraits, and based on these portraits, recommends content of the type that users like. In this way, the company retains customer traffic, increases customers’ favorable experience, and thus maximizes the product value.

industry

social interaction

Case background and challenges

Hangzhou Fuliao Technology Co., Ltd. was established in July 2013, mainly engaged in the development and operation of social applications. In terms of products, the Fuliao team has rich experience in social interaction and user behavior motivation. It always adheres to the vision of “Chatting Creates Wealth” and takes continuously improving the user experience as its ultimate goal, enabling users to obtain more fun and value in the interaction! In terms of technology, the Fuliao team focuses on video interactive social networking and has created application scenarios such as live streaming, video chatting, and short videos with high – quality user experience.

The platform will collect users’ browsing behaviors, track users’ historical data, analyze and form user profiles. According to different user profile labels, the platform will recommend content that users like. In this way, it can attract customer traffic, enhance customer experience, and improve product value and stickiness. The platform focuses on the overseas market, with the customer group mainly in Southeast Asia and some in Europe and America. If domestic computer rooms or cloud service providers are used, and the data is overseas, it is necessary to transfer the data back to China for data analysis, which will incur expensive network transmission costs. There are also certain hidden dangers in terms of security and reliability, and they cannot be guaranteed. In addition, for overseas users, the domestic computer rooms have a relatively large latency, resulting in a decline in the user experience.

According to the client’s business requirements in terms of security, reliability, performance efficiency, cost optimization, and operational excellence, formulate the client’s architecture design on GCP to enable the best practices for the Fuliao data analysis architecture, ensuring continuous improvement in security, reliability, and performance, continuous reduction in costs, as well as enhancing the user experience and industry competitiveness. Through multiple business and technical exchanges with the client, we have a better understanding of their needs. Their main requirements are “performance and cost”. Based on the above description, the business architecture design is as follows:

Scheme architecture description

Under the premise of ensuring data security, synchronize data to the Google Cloud RDS database. To this end, it is recommended that customers use the DMS service provided by GCP for real-time data synchronization between two locations; guide customers to use Cloud SQL to connect to the Alibaba Cloud source site for data migration, perform secondary storage through Bigtable, then push through the message subscription function of the Cloud Pub/Sub service, and use Cloud Dataflow for secondary cleaning to address the particularity of the source site database. Migrate Redis to Cloud Memorystore, and adopt the Cloud CDN + Cloud Load Balancing solution for global acceleration to enhance the user experience.

Use the product

• Cloud Memorystore
• Cloud SQL
• Cloud Bigtable
• Cloud Load balancing
• Cloud CDN

Features of the plan

Security:

To ensure the security of the Fuliao data analysis architecture, it needs to be implemented through a risk assessment and mitigation strategic approach, so as to provide the protection capabilities of information, systems, and assets when delivering business value. The security of the ASM service architecture can be guaranteed through practices in the following aspects.

• IAM Permission Management: To perform data analysis, one needs the Role of GCP services. Here, only access to pubsub/Dataflow/BQ/LB/SQL, as well as the planning of IAM User and MFA, is required.
  Data Transmission Security: Except for the gateway which is on the public network, everything else is on the private network. For the gateway, a GLB with an SSL-certificate-enabled Https listener needs to be configured, and an SSL certificate should be configured for the backend.
• Data storage security: GCS Bucket data encryption, local SSD data encryption.
• Host Security: Defining the access rules for the firewall of GCE and LB includes: ports, protocols, and IP addresses. Try to avoid using usernames and passwords. When performing maintenance, do not directly access through the public network. Instead, use a jump server for maintenance.
• Cybersecurity: To reduce or prevent cyberattacks, Cloud Armor and firewall rules can be adopted to prevent DDoS attacks and protect web services. Additionally, reinforcement can be carried out at the architectural level, and real-time access analysis can be conducted at the operational level. This architecture is for web services, and the access targets are agents. The access method is almost always through the Internet. Therefore, it is necessary to properly configure firewall inbound rules, use jump servers, conduct log monitoring and analysis, and focus on the architecture, etc.

Reliability:

The data analysis architecture of Fuliao must have the ability to recover from infrastructure or service failures, the ability to meet the dynamic on – demand allocation of computing resources, and the ability to avoid issues such as network outages or misconfigurations. The reliability of the Fuliao data analysis architecture can be ensured through the following three best practices:

• Fundamentals: Manage the resource limits of your account on GCP well (it is necessary to check the GCP resource limits in advance and submit a case in advance), plan the network topology well (when planning VPC and subnets, try to make preparations for future expansion), and if possible, adopt cross – regional high – availability deployment.
• Change Management: Ensure that the business can automatically adapt to on-demand modifications (enable the Auto-scaling feature for the instance group to cope with the demand for traffic), and reasonably monitor GCP resources (use Cloud Monitoring and Cloud Logging to mainly monitor basic parameters such as CPU, memory, storage, and network).
• Failure Management: Ensure backup data (implement data lifecycle management using GCS Bucket and CDN log lifecycle management), and provide periodic resilience and disaster recovery drills (conduct semi – annual simulation drills for critical businesses).

Performance efficiency:

To provide the ability to effectively use computing resources for the data analysis architecture of Geifulliao to meet system requirements, and to ensure efficiency even as requirements change and technology improves, the following four best practices can be adopted to configure performance efficiency:

• Selection: Design a rich chat data analysis architecture with optimal performance based on the best practices of GCP’s web service architecture and business requirements. This includes computing, storage, and networking (the entire CDN platform is deployed in Singapore. In line with business and business development team requirements, the standard machine n1-standard-8 is used as the main computing resource, while GCS is used for static data storage. At the same time, local SSD is used for data storage with high read and write requirements, and an L2 Cache is deployed to reduce user access latency and relieve the pressure on the origin server).
• Review: By continuously deepening the understanding of GCP services, the business development team can adopt the most reasonable resource types to meet business needs (for example: consider using Auto-scaling, etc.).
• Monitoring: Through Stackdriver, Cloud Monitoring, and Cloud Logging, you can monitor running resources in real-time, enabling you to understand the efficiency of business resource utilization. When the monitored threshold is exceeded, an alarm can be automatically sent via email to relevant administrators.
• Trade-off: To achieve higher performance, it is necessary to consider the balance between consistency, durability, space and time, and latency. Considering the requirements of different services of the Fuliao Data Analysis Platform and future SaaS considerations, adopting LB + instance group can provide better services in terms of performance, access experience, and security.

Cost Optimization:

To save costs and avoid or eliminate unnecessary costs or inefficient resources in the Fuliao Data Analysis Platform architecture, cost optimization can be configured through the following four best practices:

• Effective resource costs: When choosing resources, the size of the resources, service type, and pricing model must be considered. Ensure that Fuliao data analysis can provide a better service experience. The pricing model is deployed in an Auto-scaling manner to reduce unnecessary resource waste. Storage uses GCS and local SSDs and implements lifecycle management.
• Supply-demand matching: When allocating resources, it is necessary to consider that the quantity, region, size of resources and the pricing model are reasonable. More resources need to be prepared for promotional activities, and resources are not required to be available on weekends. Through the corresponding GCP tool services, idle resources are shut down, and appropriate preemptible virtual machines are purchased.
• Awareness of usage and expenditure: Consider traffic costs (try to use the internal network of GCP, and do not use outbound public network traffic unless necessary), monitor the rationality of resource usage and costs (use Cloud Monitoring to monitor resources in real – time), stop or shut down temporary or no – longer – needed resources, and have a standard resource application and approval process.
• Continuous Optimization: Create a cost – optimizing process: Keep track of new features and services of GCP in real – time. Review: Check new GCP services, resource types and sizes in real – time, and test and evaluate whether these resources meet the requirements of performance improvement and cost reduction.

Excellent Operations:

In order to provide best practices for the Fuliao Data Analysis Platform, one must have the ability to operate and monitor the value of the delivered business system, and be able to continuously improve the delivery process and content. Excellent operations can be achieved through the following three best practices:

• Preparation: Utilize the operational best practices on GCP cloud and the configuration management of business workloads to operate the Fuliao Data Analysis Platform on the most suitable GCP resources, optimizing security, reliability, performance, and cost. For example, not too many managed services will be selected in the early stage, as this will increase the development workload. They will be gradually replaced as the use of GCP deepens.
• Operation and Maintenance: Increase the business load with minimal configuration changes; monitor your business to ensure it can operate as expected.
• Response: Prepare for both unplanned and planned responses, and arrange the service level SLA and SOW of the Rich Chat Data Analysis Platform.

Customer benefits

• After using Cloud SQL, the database availability has been increased to 99.95%; the interface response time has been improved by 500%.
• The user portraits drawn with the help of BigQuery and machine learning have significantly increased the user retention rate, and the daily active users have increased by 32%.
• With the service architecture integrated by auto-scaling, the cost of backend cloud computing services is reduced by 30 – 40%.