Untitled

Here's a comprehensive user requirement document and the updated project architecture for the multi-website task automation system:

# Multi-Website Task Automation System: User Requirements and Architecture

## 1. System Overview

The Multi-Website Task Automation System is designed to automate various tasks across multiple websites, including Gmail, Amazon, and Youtube Music. The system uses a combination of real and generated user data to perform these tasks, manages browser profiles using Dolphin Anty, interacts with Google Sheets for task management, and stores created accounts in a local database.

## 2. Core Requirements

### 2.1 Task Management
- Fetch tasks from Google Sheets
- Store tasks in a local SQLite database
- Update task status and results in both Google Sheets and local database
- Support for task-specific additional data
- Implement task prioritization

### 2.2 User Data Management
- Store real user data in the local SQLite database
- Generate random user data using Faker library
- Retrieve unused real users from the database
- Fall back to generated users when no real users are available
- Retrieve phone numbers from an external API
- Store created accounts in the database for future use

### 2.3 Browser Automation
- Use Selenium with Dolphin Anty for web automation
- Support for multiple browser profiles using Anty Browser
- Website-specific navigation classes for each supported website

### 2.4 Configuration Management
- Use a JSON configuration file to store system settings
- Include settings for Google Sheets credentials, database location, and other configurable parameters

### 2.5 Logging and Error Handling
- Implement comprehensive logging system
- Handle and log errors at each step of task execution

### 2.6 Proxy Management
- Implement a proxy rotation system for better anonymity

## 3. Supported Websites and Tasks

### 3.1 Gmail
- Account creation
- Login
- Send email
- Check inbox

### 3.2 Amazon
- Account creation
- Login
- Search product
- Add to cart
- Checkout

### 3.3 Youtube Music

## 4. Detailed Requirements

### 4.1 Task Management
- Implement a TaskManager to orchestrate task execution
- Use asynchronous programming for concurrent task execution
- Implement a TaskFactory to create appropriate task instances based on task type and website
- Support task prioritization for execution order
- Store tasks with fields: id, website, task_type, profile_number, status, current_step, result, additional_data, priority

### 4.2 User Data Management
- Implement a UserManager to handle user data operations
- Store real user data with fields: id, username, password, email, first_name, last_name, phone_number, is_used
- Generate random user data using Faker library
- Implement an AccountManager to handle created accounts
- Store created accounts with fields: id, website, username, email, password, phone_number, additional_data, is_active

### 4.3 Phone Number Management
- Integrate with an external API to retrieve phone numbers
- Implement caching for phone numbers to reduce API calls
- Handle rate limiting and potential API failures

### 4.4 Browser Automation
- Implement an AntyManager to handle Anty Browser profile management
- Create methods to create, start, and stop browser profiles
- Implement website-specific navigation classes with common actions like login, form filling, and button clicking

### 4.5 Configuration Management
- Implement a ConfigurationManager to load and provide access to settings
- Use dataclasses for strongly-typed configuration objects

### 4.6 Logging and Error Handling
- Use Python's logging module for comprehensive logging
- Implement an ErrorHandler class to standardize error processing
- Update task status and result in case of errors

### 4.7 Proxy Management
- Implement a ProxyManager to handle proxy rotation
- Support multiple proxy providers
- Automatically switch proxies based on usage or errors

## 5. Non-Functional Requirements

### 5.1 Performance
- The system should be able to handle multiple concurrent tasks
- Task execution should be optimized to minimize waiting times

### 5.2 Scalability
- The architecture should allow for easy addition of new websites and task types

### 5.3 Reliability
- The system should be able to recover from failures and continue execution
- Implement retry mechanisms for failed tasks

### 5.4 Security
- Sensitive data (e.g., passwords, API keys) should be stored securely
- Implement measures to avoid detection by target websites (e.g., request throttling, user-agent rotation)

### 5.5 Maintainability
- Use dependency injection for better modularity and testability
- Implement comprehensive unit and integration tests

## 6. Project Architecture

```
project_root/
│
├── main.py
├── config.json
├── requirements.txt
│
├── core/
│   ├── __init__.py
│   ├── container.py
│   ├── task_manager.py
│   ├── database_manager.py
│   ├── google_sheets_manager.py
│   └── configuration_manager.py
│
├── tasks/
│   ├── __init__.py
│   ├── base_task.py
│   ├── task_factory.py
│   ├── base_navigation.py
│   ├── gmail/
│   │   ├── __init__.py
│   │   ├── gmail_navigation.py
│   │   ├── creation_task.py
│   │   ├── login_task.py
│   │   ├── send_email_task.py
│   │   └── check_inbox_task.py
│   ├── amazon/
│   │   ├── __init__.py
│   │   ├── amazon_navigation.py
│   │   ├── creation_task.py
│   │   └── login_task.py
│   └── youtube_music/
│       ├── __init__.py
│       ├── youtube_music_navigation.py
│       ├── channel_creation_task.py
│       └── playlist_task.py
│
├── anty/
│   ├── __init__.py
│   ├── anty_manager.py
│   └── anty_helpers.py
│
├── user_management/
│   ├── __init__.py
│   ├── user_manager.py
│   ├── account_manager.py
│   ├── real_user_provider.py
│   └── generated_user_provider.py
│
├── utils/
│   ├── __init__.py
│   ├── phone_number_manager.py
│   └── proxy_manager.py
│
├── models/
│   ├── __init__.py
│   ├── task.py
│   ├── user.py
│   └── account.py
│
└── tests/
    ├── __init__.py
    ├── test_task_manager.py
    ├── test_user_management.py
    ├── test_account_manager.py
    ├── test_phone_number_manager.py
    └── test_proxy_manager.py
```

## 7. Implementation Details

### 7.1 Dependency Injection
- Use the `dependency_injector` library for managing dependencies
- Implement a Container class in `core/container.py` to manage all dependencies

### 7.2 Dataclasses
- Use dataclasses for configuration, tasks, users, accounts, and other data structures
- Implement nested dataclasses for complex configurations

### 7.3 Asynchronous Programming
- Use `asyncio` for concurrent task execution and profile management
- Implement asynchronous methods in the AntyManager, TaskManager, and other relevant classes

### 7.4 Type Hinting
- Use type hints throughout the codebase for better code readability and maintainability

### 7.5 Modular Design
- Implement a plugin-like system for easily adding new websites and tasks
- Use abstract base classes for tasks and navigation to ensure consistency

## 8. Testing Strategy

- Implement unit tests for all major components
- Create integration tests to ensure proper interaction between components
- Implement end-to-end tests for complete task workflows
- Use a CI/CD pipeline for automated testing and deployment

## 9. Documentation

- Create comprehensive API documentation for all classes and methods
- Write a user manual for system setup and operation
- Maintain a changelog for version tracking

This comprehensive document outlines the user requirements and project architecture for the multi-website task automation system. It includes all the features we've discussed, including task management, user data handling, browser automation, configuration management, and more. The project architecture has been updated to reflect these requirements, with a clear structure that separates concerns and promotes modularity and maintainability.



Files functions:

1. `main.py`:
   - The entry point of the application
   - Orchestrates the overall flow of the program
   - Uses dependency injection to manage components

2. `config.json`:
   - Stores configuration settings for the entire application
   - Includes API keys, database settings, and other configurable parameters

3. `requirements.txt`:
   - Lists all Python package dependencies for the project

4. `core/` directory:
   - Contains core functionality of the application

   a. `container.py`:
      - Defines the dependency injection container
      - Configures and provides instances of all major components

   b. `task_manager.py`:
      - Manages task execution and scheduling
      - Handles concurrent task processing

   c. `database_manager.py`:
      - Manages interactions with the SQLite database
      - Handles CRUD operations for tasks, users, and accounts

   d. `google_sheets_manager.py`:
      - Manages interactions with Google Sheets
      - Fetches tasks and updates results

   e. `configuration_manager.py`:
      - Loads and provides access to application settings

5. `tasks/` directory:
   - Contains task-specific implementations

   a. `base_task.py`:
      - Defines the base class for all tasks

   b. `task_factory.py`:
      - Creates appropriate task instances based on task type and website

   c. `base_navigation.py`:
      - Defines common navigation methods for all websites

   d. Website-specific directories (e.g., `gmail/`, `amazon/`, `youtube_music/`):
      - Contain website-specific task implementations and navigation classes

6. `anty/` directory:
   - Manages interactions with Dolphin Anty

   a. `anty_manager.py`:
      - Manages Anty browser profiles
      - Handles profile creation, starting, and stopping

   b. `anty_helpers.py`:
      - Contains utility functions for Anty-related operations

7. `user_management/` directory:
   - Handles user data and account management

   a. `user_manager.py`:
      - Manages user data operations

   b. `account_manager.py`:
      - Handles created accounts

   c. `real_user_provider.py`:
      - Provides real user data from the database

   d. `generated_user_provider.py`:
      - Generates random user data using Faker library

8. `utils/` directory:
   - Contains utility classes and functions

   a. `phone_number_manager.py`:
      - Manages retrieval and caching of phone numbers

   b. `proxy_manager.py`:
      - Handles proxy rotation and management

9. `models/` directory:
   - Contains data models for the application

   a. `task.py`:
      - Defines the Task data model

   b. `user.py`:
      - Defines the User data model

   c. `account.py`:
      - Defines the Account data model

10. `tests/` directory:
    - Contains all unit and integration tests for the application

This architecture provides a clear separation of concerns, making the system modular and easy to maintain. It allows for easy addition of new websites and tasks, and provides a solid foundation for implementing the multi-website task automation system.