159 lines
7.5 KiB
Markdown
159 lines
7.5 KiB
Markdown
# Architecture
|
|
|
|
Marketing Parser is a single Spring Boot 3.5.5 service on Java 21, backed by MongoDB
|
|
for persistence and MinIO for generated artefacts. It exposes a REST API consumed by
|
|
the KonturAI frontend and runs a set of scheduled background jobs.
|
|
|
|
## Functional areas
|
|
|
|
The service covers five loosely coupled concerns that share a database and a set of
|
|
AI clients:
|
|
|
|
1. **RSS ingestion** — collects business news into a `MarketItem` corpus.
|
|
2. **Marketing analysis** — turns that corpus plus user input into structured analysis.
|
|
3. **Report rendering** — renders analysis as PDF, DOCX and Markdown with charts.
|
|
4. **Campaign execution** — generates creatives and publishes them to social networks.
|
|
5. **Targeting and leads** — ad targeting recommendations and Facebook lead capture.
|
|
|
|
## Request flow
|
|
|
|
```
|
|
Frontend
|
|
│ Bearer JWT
|
|
▼
|
|
Controller ──validator──► DTO
|
|
│ │
|
|
│ ▼
|
|
│ Service layer
|
|
│ ╱ │ ╲
|
|
│ AI clients Repository MinIO
|
|
│ (OpenAI/Ollama/ │ (artefacts)
|
|
│ Vertex/Serper) ▼
|
|
│ MongoDB
|
|
▼
|
|
GlobalExceptionHandler ──► consistent error payload
|
|
```
|
|
|
|
Authentication is **JWT bearer tokens issued by a separate auth service**. This
|
|
service does not log users in; `JwtService` validates the signature against
|
|
`security.jwt.secret-base64` and extracts the user id, email and roles from claims.
|
|
Controllers pull the caller identity via `extractUserIdFromHeader(authHeader)`. See
|
|
[api/authentication.md](api/authentication.md).
|
|
|
|
## Layers
|
|
|
|
| Package | Responsibility |
|
|
| --- | --- |
|
|
| `controller` | HTTP endpoints, 15 controllers plus `GlobalExceptionHandler` |
|
|
| `validator` | Request-level validation beyond Bean Validation annotations |
|
|
| `dto` | Request/response payloads, including `dto/targeting` |
|
|
| `service` | All business logic — 52 classes |
|
|
| `repository` | Spring Data MongoDB interfaces |
|
|
| `model` | MongoDB documents |
|
|
| `config` | Beans, typed `@ConfigurationProperties`, async executors, CORS, Swagger |
|
|
| `exception` | Domain exceptions surfaced by `GlobalExceptionHandler` |
|
|
|
|
## Endpoints
|
|
|
|
| Base path | Controller | Purpose |
|
|
| --- | --- | --- |
|
|
| `/api/parser/health` | `HealthCheckController` | Liveness |
|
|
| `/api/parser/items` | `MarketItemController` | Access the ingested news corpus |
|
|
| `/api/parser/admin/parsers` | `ParserAdminController` | Trigger and inspect parsers |
|
|
| `/api/parser/report` | `ReportController` | Research report generation and history |
|
|
| `/api/marketing/analysis` | `MarketingController` | Marketing analysis (v1/v2) |
|
|
| `/api/marketing/v3` | `MarketingAnalysisV3Controller` | Marketing analysis v3 |
|
|
| `/api/marketing/targeting` | `TargetingCampaignController` | Campaign targeting |
|
|
| `/api/marketing` | `PublicAssetController` | Public access to generated assets |
|
|
| `/api/targeting` | `AiTargetingSystemController` | AI targeting recommendations |
|
|
| `/api/social-media/credentials` | `SocialMediaCredentialsController` | Per-user network credentials |
|
|
| `/api/facebook/config` | `FacebookConfigController` | Facebook app/page configuration |
|
|
| `/api/facebook/leads` | `FacebookLeadsController` | Collected hot leads |
|
|
| `/api/facebook/webhook` | `FacebookWebhookController` | Facebook webhook receiver |
|
|
| `/api/openai` | `OpenAITestController` | Connectivity diagnostics |
|
|
|
|
Full request/response detail: [api/README.md](api/README.md), or the live OpenAPI UI
|
|
at `/swagger-ui.html`.
|
|
|
|
## RSS ingestion
|
|
|
|
`ParserService` is a small interface — `getSourceName()` and `parseAndSaveRssFeed()`.
|
|
Each source implements it, and `ParserManagerService` acts as a facade: Spring injects
|
|
every `ParserService` bean and the manager indexes them by source name, so adding a
|
|
source requires no changes to the manager or the controller.
|
|
|
|
Five sources are implemented: Kursiv, Kapital, LSM, RBC and Vedomosti. Details and
|
|
scheduling in [rss-parsers.md](rss-parsers.md).
|
|
|
|
## Scheduled jobs
|
|
|
|
`parser.scheduler.enabled` is **`false` by default** — RSS polling does not run unless
|
|
explicitly enabled. `posting.scheduler.enabled` defaults to `true`.
|
|
|
|
| Job | Cron | Owner |
|
|
| --- | --- | --- |
|
|
| Kursiv / Kapital ingest | every 5 min | `KursivParserService`, `KapitalParserService` |
|
|
| LSM / RBC / Vedomosti ingest | every 30 min | respective parser services |
|
|
| Scheduled post publishing | every minute | `PostingSchedulerService` |
|
|
| Facebook lead collection | every 15 min (configurable) | `FacebookLeadCollectorService` |
|
|
| Targeting campaign sync | every 6 hours | `TargetingCampaignService` |
|
|
| Campaign prediction refresh | daily 06:00 | `CampaignPredictionService` |
|
|
|
|
## AI provider strategy
|
|
|
|
The service deliberately mixes providers by cost and capability:
|
|
|
|
- **OpenAI** (`gpt-4o`, `gpt-4o-mini`) — structured JSON analysis and chart data, where
|
|
reliable schema adherence matters. Wrapped in retry with exponential backoff and a
|
|
concurrency cap of 3.
|
|
- **Ollama** (self-hosted) — long-form narrative text, avoiding per-token cost on the
|
|
largest outputs. Timeouts are correspondingly long (up to 5 hours).
|
|
- **Google Vertex AI** — Imagen 3 for images, Veo 3 for video, authenticated with a
|
|
service-account key. See [configuration.md](configuration.md#the-google-service-account-key).
|
|
- **Serper** — Google search results feeding research reports.
|
|
|
|
`ClaudeApiService` and `DeepResearchService` cover additional generation paths.
|
|
|
|
## Persistence
|
|
|
|
MongoDB documents, one repository each:
|
|
|
|
| Document | Holds |
|
|
| --- | --- |
|
|
| `MarketItem` | Ingested news articles (the corpus) |
|
|
| `MarketingAnalysis`, `MarketingAnalysisV2Document`, `MarketingAnalysisV3Document` | Three analysis generations, kept side by side |
|
|
| `MarketingStrategy` | Generated promotion strategies |
|
|
| `TargetingCampaign`, `TargetingAudienceProfile`, `TargetingAdSet`, `TargetingAd`, `TargetingInsight` | Ad targeting model |
|
|
| `PostingTask` | Queued and published social posts |
|
|
| `SocialMediaCredentials` | Per-user network credentials, encrypted at rest |
|
|
| `FacebookLead` | Leads harvested from page comments |
|
|
| `ReportHistory` | Generated report metadata |
|
|
| `CampaignPrediction`, `ABTestConfig`, `BudgetConfig`, `PerformanceMetrics` | Campaign optimisation |
|
|
|
|
The three analysis document versions coexist because the API kept older revisions
|
|
working for the frontend; `MarketingAnalysisV3Service` is the current path.
|
|
|
|
## External dependencies
|
|
|
|
| Dependency | Used for | Failure behaviour |
|
|
| --- | --- | --- |
|
|
| MongoDB | All persistence | Fatal — service cannot operate |
|
|
| MinIO | Reports and generated images | Feature-level failure |
|
|
| OpenAI | Analysis, strategy, chart data | Retried, then surfaced as an error |
|
|
| Ollama | Long-form report text | Retried, then surfaced as an error |
|
|
| Vertex AI | Image/video generation | Logs error, returns `null`; rest of the service unaffected |
|
|
| Serper | Research search | Retried |
|
|
| Facebook Graph API | Posting and lead collection | Logged, retried on next schedule |
|
|
| SMTP | Emailing reports | Health indicator disabled; failures logged |
|
|
|
|
## Cross-cutting configuration
|
|
|
|
- **CORS** — `CorsProperties` binds `cors.*`, applied by `WebCorsConfig`. Origins
|
|
default to the production frontend domains.
|
|
- **Async** — `AsyncConfig` and `TargetingAsyncConfig` provide executors; long AI calls
|
|
run off the request thread and parsers return `CompletableFuture`.
|
|
- **Error handling** — `GlobalExceptionHandler` maps domain exceptions to consistent
|
|
payloads.
|
|
- **API docs** — `SwaggerConfig` declares the `bearerAuth` scheme so the Swagger UI can
|
|
authorise with a JWT.
|