Skip to content

Commit 65972f2

Browse files
TheodoreSpeaksTheodoreSpeaks
authored
fix(retention): switch data retention to be org-level (#4270)
* fix(retention): switch data retention to be org-level * fix lint * cleanup mothership ran logs * fix cleanup dispatcher * fix ui flash for data retention settings * fix lint * remove raw sql string interprolation
1 parent 5f0f0ed commit 65972f2

11 files changed

Lines changed: 15600 additions & 290 deletions

File tree

‎apps/docs/content/docs/en/enterprise/data-retention.mdx‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ description: Control how long execution logs, deleted resources, and copilot dat
66
import { FAQ } from '@/components/ui/faq'
77
import { Image } from '@/components/ui/image'
88

9-
Data Retention lets workspace admins on Enterprise plans configure how long three categories of data are kept before they are permanently deleted. Each workspace in your organization can have its own independent configuration.
9+
Data Retention lets organization owners and admins on Enterprise plans configure how long three categories of data are kept before they are permanently deleted. The configuration applies to every workspace in the organization.
1010

1111
---
1212

@@ -58,9 +58,9 @@ Each setting is independent. You can configure a short log retention period alon
5858

5959
---
6060

61-
## Per-workspace configuration
61+
## Organization-wide configuration
6262

63-
Retention is configured at the **workspace level**, not organization-wide. Each workspace in your organization can have a different configuration. Changes to one workspace's settings do not affect other workspaces.
63+
Retention is configured at the **organization level**. A single configuration applies to every workspace in the organization — there are no per-workspace overrides.
6464

6565
---
6666

@@ -73,7 +73,7 @@ By default, all three settings are unconfigured — no data is automatically del
7373
<FAQ items={[
7474
{
7575
question: "Who can configure data retention settings?",
76-
answer: "Only workspace admins can configure data retention settings. On Sim Cloud, the workspace must be on an Enterprise plan."
76+
answer: "Only organization owners and admins can configure data retention settings. On Sim Cloud, the organization must be on an Enterprise plan."
7777
},
7878
{
7979
question: "Is deletion immediate once the retention period expires?",
@@ -85,7 +85,7 @@ By default, all three settings are unconfigured — no data is automatically del
8585
},
8686
{
8787
question: "Does the retention period apply to all workspaces in my organization?",
88-
answer: "No. Retention is configured per workspace. Each workspace in your organization can have a different configuration."
88+
answer: "Yes. Retention is configured once per organization and applies to every workspace in the organization."
8989
},
9090
{
9191
question: "What happens if I shorten the retention period?",

‎apps/sim/app/api/organizations/[id]/data-retention/route.ts‎

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
import { AuditAction, AuditResourceType, recordAudit } from '@sim/audit'
2+
import { db } from '@sim/db'
3+
import { member, organization } from '@sim/db/schema'
4+
import { createLogger } from '@sim/logger'
5+
import { and, eq } from 'drizzle-orm'
6+
import { type NextRequest, NextResponse } from 'next/server'
7+
import { z } from 'zod'
8+
import { getSession } from '@/lib/auth'
9+
import {
10+
CLEANUP_CONFIG,
11+
type OrganizationRetentionSettings,
12+
} from '@/lib/billing/cleanup-dispatcher'
13+
import { isOrganizationOnEnterprisePlan } from '@/lib/billing/core/subscription'
14+
import { isBillingEnabled } from '@/lib/core/config/feature-flags'
15+
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
16+
17+
const logger = createLogger('DataRetentionAPI')
18+
19+
const MIN_HOURS = 24
20+
const MAX_HOURS = 43800
21+
22+
const updateRetentionSchema = z.object({
23+
logRetentionHours: z.number().int().min(MIN_HOURS).max(MAX_HOURS).nullable().optional(),
24+
softDeleteRetentionHours: z.number().int().min(MIN_HOURS).max(MAX_HOURS).nullable().optional(),
25+
taskCleanupHours: z.number().int().min(MIN_HOURS).max(MAX_HOURS).nullable().optional(),
26+
})
27+
28+
function enterpriseDefaults(): OrganizationRetentionSettings {
29+
return {
30+
logRetentionHours: CLEANUP_CONFIG['cleanup-logs'].defaults.enterprise,
31+
softDeleteRetentionHours: CLEANUP_CONFIG['cleanup-soft-deletes'].defaults.enterprise,
32+
taskCleanupHours: CLEANUP_CONFIG['cleanup-tasks'].defaults.enterprise,
33+
}
34+
}
35+
36+
function normalizeConfigured(
37+
settings: Partial<OrganizationRetentionSettings> | null | undefined
38+
): OrganizationRetentionSettings {
39+
return {
40+
logRetentionHours: settings?.logRetentionHours ?? null,
41+
softDeleteRetentionHours: settings?.softDeleteRetentionHours ?? null,
42+
taskCleanupHours: settings?.taskCleanupHours ?? null,
43+
}
44+
}
45+
46+
/**
47+
* GET /api/organizations/[id]/data-retention
48+
* Returns the organization's data retention settings.
49+
* Accessible by any member of the organization.
50+
*/
51+
export const GET = withRouteHandler(
52+
async (_request: NextRequest, { params }: { params: Promise<{ id: string }> }) => {
53+
const session = await getSession()
54+
if (!session?.user?.id) {
55+
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
56+
}
57+
58+
const { id: organizationId } = await params
59+
60+
const [memberEntry] = await db
61+
.select({ id: member.id })
62+
.from(member)
63+
.where(and(eq(member.organizationId, organizationId), eq(member.userId, session.user.id)))
64+
.limit(1)
65+
66+
if (!memberEntry) {
67+
return NextResponse.json(
68+
{ error: 'Forbidden - Not a member of this organization' },
69+
{ status: 403 }
70+
)
71+
}
72+
73+
const [org] = await db
74+
.select({ dataRetentionSettings: organization.dataRetentionSettings })
75+
.from(organization)
76+
.where(eq(organization.id, organizationId))
77+
.limit(1)
78+
79+
if (!org) {
80+
return NextResponse.json({ error: 'Organization not found' }, { status: 404 })
81+
}
82+
83+
const isEnterprise = !isBillingEnabled || (await isOrganizationOnEnterprisePlan(organizationId))
84+
const configured = normalizeConfigured(org.dataRetentionSettings)
85+
const defaults = enterpriseDefaults()
86+
87+
return NextResponse.json({
88+
success: true,
89+
data: {
90+
isEnterprise,
91+
defaults,
92+
configured,
93+
effective: isEnterprise ? configured : defaults,
94+
},
95+
})
96+
}
97+
)
98+
99+
/**
100+
* PUT /api/organizations/[id]/data-retention
101+
* Updates the organization's data retention settings.
102+
* Requires enterprise plan and owner/admin role.
103+
*/
104+
export const PUT = withRouteHandler(
105+
async (request: NextRequest, { params }: { params: Promise<{ id: string }> }) => {
106+
const session = await getSession()
107+
if (!session?.user?.id) {
108+
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
109+
}
110+
111+
const { id: organizationId } = await params
112+
113+
const body = await request.json()
114+
const parsed = updateRetentionSchema.safeParse(body)
115+
if (!parsed.success) {
116+
return NextResponse.json(
117+
{ error: parsed.error.errors[0]?.message ?? 'Invalid request body' },
118+
{ status: 400 }
119+
)
120+
}
121+
122+
const [memberEntry] = await db
123+
.select({ role: member.role })
124+
.from(member)
125+
.where(and(eq(member.organizationId, organizationId), eq(member.userId, session.user.id)))
126+
.limit(1)
127+
128+
if (!memberEntry) {
129+
return NextResponse.json(
130+
{ error: 'Forbidden - Not a member of this organization' },
131+
{ status: 403 }
132+
)
133+
}
134+
135+
if (memberEntry.role !== 'owner' && memberEntry.role !== 'admin') {
136+
return NextResponse.json(
137+
{ error: 'Forbidden - Only organization owners and admins can update data retention' },
138+
{ status: 403 }
139+
)
140+
}
141+
142+
if (isBillingEnabled) {
143+
const hasEnterprise = await isOrganizationOnEnterprisePlan(organizationId)
144+
if (!hasEnterprise) {
145+
return NextResponse.json(
146+
{ error: 'Data Retention is available on Enterprise plans only' },
147+
{ status: 403 }
148+
)
149+
}
150+
}
151+
152+
const [currentOrg] = await db
153+
.select({
154+
name: organization.name,
155+
dataRetentionSettings: organization.dataRetentionSettings,
156+
})
157+
.from(organization)
158+
.where(eq(organization.id, organizationId))
159+
.limit(1)
160+
161+
if (!currentOrg) {
162+
return NextResponse.json({ error: 'Organization not found' }, { status: 404 })
163+
}
164+
165+
const current = normalizeConfigured(currentOrg.dataRetentionSettings)
166+
const merged: OrganizationRetentionSettings = { ...current }
167+
if (parsed.data.logRetentionHours !== undefined) {
168+
merged.logRetentionHours = parsed.data.logRetentionHours
169+
}
170+
if (parsed.data.softDeleteRetentionHours !== undefined) {
171+
merged.softDeleteRetentionHours = parsed.data.softDeleteRetentionHours
172+
}
173+
if (parsed.data.taskCleanupHours !== undefined) {
174+
merged.taskCleanupHours = parsed.data.taskCleanupHours
175+
}
176+
177+
const [updated] = await db
178+
.update(organization)
179+
.set({ dataRetentionSettings: merged, updatedAt: new Date() })
180+
.where(eq(organization.id, organizationId))
181+
.returning({ dataRetentionSettings: organization.dataRetentionSettings })
182+
183+
if (!updated) {
184+
return NextResponse.json({ error: 'Organization not found' }, { status: 404 })
185+
}
186+
187+
recordAudit({
188+
workspaceId: null,
189+
actorId: session.user.id,
190+
action: AuditAction.ORGANIZATION_UPDATED,
191+
resourceType: AuditResourceType.ORGANIZATION,
192+
resourceId: organizationId,
193+
actorName: session.user.name ?? undefined,
194+
actorEmail: session.user.email ?? undefined,
195+
resourceName: currentOrg.name,
196+
description: 'Updated data retention settings',
197+
metadata: { changes: parsed.data },
198+
request,
199+
})
200+
201+
const configured = normalizeConfigured(updated.dataRetentionSettings)
202+
const defaults = enterpriseDefaults()
203+
204+
return NextResponse.json({
205+
success: true,
206+
data: {
207+
isEnterprise: true,
208+
defaults,
209+
configured,
210+
effective: configured,
211+
},
212+
})
213+
}
214+
)

0 commit comments

Comments
 (0)