Settings

Organization settings allow you to configure your workspace name, LLM providers, and other preferences. This guide covers all available settings and how to manage them.

Accessing Settings

Navigate to Settings in one of two ways:

• Click the gear icon in the navigation
• Go directly to /{org-slug}/settings

Settings Tabs Overview

General Settings

Organization Name

The display name for your organization. It appears in:

• The organization switcher
• Member lists
• Invitation emails

To change the name:

1. Go to Settings → General
2. Edit the Organization Name field
3. Click Save

Note: Only Owners can change the organization name.

Organization Slug

The URL-friendly identifier for your organization (e.g., acme-team).

Important: The slug cannot be changed after creation. Choose carefully when creating an organization.

The slug appears in all URLs:

https://app.llmx.io/acme-team/
https://app.llmx.io/acme-team/settings
https://app.llmx.io/acme-team/prompts/...

Default Member Role

The pre-selected role when inviting new members.

You can always change the role for individual invitations.

LLM Provider Configuration

Configure API keys for the LLM providers your organization uses. These keys power the “Lab” testing feature and other AI-powered functionality.

Supported Providers

Adding a Provider API Key

1. Go to Settings → Providers
2. Find the provider you want to configure
3. Click Add Key or the configure button
4. Enter your API key
5. Click Save

The system will:

• Validate the key by making a test request
• Show a success message if valid
• Display an error if the key is invalid

Key Security

Provider API keys are stored with enterprise-grade security:

• Encrypted at rest using Fernet (AES-128)
• Never displayed after saving
• Only a masked hint is shown (e.g., sk-...7a2b)

Updating a Provider Key

1. Go to Settings → Providers
2. Find the provider
3. Click Update Key
4. Enter the new key
5. Click Save

The old key is immediately replaced.

Removing a Provider Key

1. Go to Settings → Providers
2. Find the provider
3. Click Remove
4. Confirm the removal

Warning: Removing a provider key will prevent Lab testing with that provider’s models.

Model Whitelisting

Owner only feature

Restrict which models are available to your team. Useful for:

• Cost control (prevent expensive models)
• Compliance (only approved models)
• Simplicity (reduce model choice confusion)

Enabling Model Restrictions

1. Go to Settings → Providers
2. Select a provider
3. Toggle Restrict Models
4. Check the models you want to allow
5. Click Save

Modes

When in whitelist mode, users can only select from the enabled models in the Lab and deployment settings.

Danger Zone

The Danger Zone contains irreversible actions. Access is restricted to Owners only.

Deleting an Organization

Permanently delete your organization and all its data.

What gets deleted:

• All prompts and folders
• All versions and deployments
• All team members and invitations
• All API keys
• All LLM provider configurations
• All test results and history

To delete an organization:

1. Go to Settings → Danger Zone
2. Click Delete Organization
3. Type the organization name to confirm
4. Click Delete Forever

Warning: This action is irreversible. All data is permanently deleted and cannot be recovered. Make sure to export any important content before deleting.

Confirmation Requirement

To prevent accidental deletion, you must:

1. Type the exact organization name
2. The delete button only enables when the name matches

What Happens After Deletion

• You’re redirected to another organization you belong to
• If you have no other organizations, a new personal workspace is created
• All members are notified (if email notifications are enabled)
• Pending invitations are cancelled

Exporting Data

Before making major changes or deleting an organization, consider exporting your data.

Export Options

Go to Settings → Export to:

• Export all prompts as JSON
• Export folder structure
• Download version history

This ensures you have a backup before irreversible changes.

For Developers

Provider Key Encryption

Provider API keys are encrypted using Fernet symmetric encryption:

from cryptography.fernet import Fernet
 
# Encryption key from environment (never hardcoded)
ENCRYPTION_KEY = os.environ["ENCRYPTION_KEY"]
fernet = Fernet(ENCRYPTION_KEY)
 
def encrypt_api_key(key: str) -> str:
    return fernet.encrypt(key.encode()).decode()
 
def decrypt_api_key(encrypted: str) -> str:
    return fernet.decrypt(encrypted.encode()).decode()

Key Validation

Before storing, keys are validated with a test request:

async def validate_openai_key(key: str) -> bool:
    try:
        client = OpenAI(api_key=key)
        # Minimal request to verify key
        await client.models.list()
        return True
    except AuthenticationError:
        return False

Model Whitelist Structure

class ModelWhitelist(BaseModel):
    providers: Dict[str, ProviderConfig]
    updated_at: datetime
    updated_by: str
 
class ProviderConfig(BaseModel):
    mode: Literal["all", "whitelist"]
    enabled: List[str]  # Model IDs when mode is "whitelist"

Firestore structure:

organizations/{orgId}/settings/model_whitelist
├── providers: {
│   "openai": {
│     "mode": "whitelist",
│     "enabled": ["gpt-4", "gpt-3.5-turbo"]
│   },
│   "anthropic": {
│     "mode": "all",
│     "enabled": []
│   }
│ }
├── updated_at: timestamp
└── updated_by: "user_abc123"

Organization Deletion

Deletion is a recursive operation that removes all subcollections:

async def delete_organization(org_id: str):
    org_ref = db.collection("organizations").document(org_id)
 
    # Delete all subcollections
    subcollections = [
        "members", "invitations", "prompts",
        "folders", "deployments", "api_keys",
        "secrets", "settings"
    ]
 
    for subcol in subcollections:
        await delete_collection(org_ref.collection(subcol))
 
    # Delete the organization document
    await org_ref.delete()
 
async def delete_collection(collection_ref):
    docs = await collection_ref.list_documents()
    for doc in docs:
        # Recursively delete nested subcollections
        for subcol in await doc.collections():
            await delete_collection(subcol)
        await doc.delete()

Settings Access Control

# Settings read access - all members
@app.get("/api/v1/organizations/{org_id}/settings")
async def get_settings(
    org_id: str,
    token: dict = Depends(verify_firebase_token)
):
    await verify_org_member(org_id, token["uid"])
    return await org_service.get_settings(org_id)
 
# Settings write access - owner only
@app.patch("/api/v1/organizations/{org_id}/settings")
async def update_settings(
    org_id: str,
    settings: OrganizationSettings,
    token: dict = Depends(verify_firebase_token)
):
    await require_owner(org_id, token["uid"])
    return await org_service.update_settings(org_id, settings)