How to create an user via the pre_login_hook when using OIDC

[Shaltz]

Hi all,

fist all, thanks Drakkan for all your work. SFTPGo is a great tool :) :)

I'm running some test to make sure it fulfills all our needs before buying an
instance running SFTPGo on AWS marketplace.

So far, I've been able to get it working with AWS Cognito as an IdP via OIDC (AzureAD => Cognito => SFTPGo).
Now, I'd love to be able to create the users as they log in (if the OIDC connection is succesfull).

Looking at the documentation, it seems that using the pre_login_hook is the way to go.
But, for the life of me, I haven't been able to get this to work.
I couldn't find any example of a script (I want to use a script, not an API/URL for this)
that would show an example of a user creation, nor I can found what the mandatory fields, for creating an user, are....

The doc points to the OpenAPI schema to find the structure of a SFTPGo user, I am probably not reading it properly,
but I couldn't find anything usable :/ (I went through the 6000 lines a couple of time.... my brain hurts ^^).
There is something that could be what I am looking for at line 5243, but all fields seem to be mandatory,
even the "id" one. I don't understand how I could know the id of an user before it's created....
I'm a bit lost here... :/

Can someone point me in a direction of an script example (bash/python/go... whatever ^^)
that would create a user or at least, tell me what the mandatory fields are ?

thanks in advance ;)

[drakkan]

Hi,

thanks for evalutaing SFTPGo. Most of the fields are optional, you are right the OpenAPI schema does not specify mandatory ones.

You can find a minimal example here. This should be enough to get you started. After subscribing to an AWS offering, you can write to the email found on the AWS marketplace page for basic support questions like this one. Please always include your AWS ID, I don't answer if your usage is very low, I'm sorry. I have had disappointing experiences in the past. Thanks

[drakkan]

Here is a minimal example for an S3 user. I assume you have AWS credentials from the environment. If not, you must also define the AWS credentials when creating the user

{
  "status": 1,
  "username": "user1",
  "password": "password",
  "permissions": {
    "/": [
      "*"
    ]
  },
  "filesystem": {
    "provider": 1,
    "s3config": {
      "bucket": "your bucket",
      "region": "your region",
      "key_prefix": "users/user1/"
    }
  }
}

A such user will be restricted to the users/user1 prefix.

Instead of reading the OpenAPI specification from the yaml file, I suggest opening http://<sftpgo_ip>:8080/openapi/swagger-ui/ in your browser and testing the API using swagger UI.

[Shaltz]

thanks for your very quick answer Drakkan,
very much appreciated ;)

I'll give this a try in the next couple of days ;)

have a great weekend :)

[Shaltz]

@drakkan
I'm thinking of something...
I realise that you must have a todo list as long as a month of Sundays,
but, in your opinion, could the EventManager be leveraged to create users
on OIDC login events ?

Just to have your thoughts on this, not to request for a new feature ;)

thanks again for everything you do ;)

[drakkan]

@drakkan I'm thinking of something... I realise that you must have a todo list as long as a month of Sundays, but, in your opinion, could the EventManager be leveraged to create users on OIDC login events ?

Just to have your thoughts on this, not to request for a new feature ;)

thanks again for everything you do ;)

perhaps by leveraging groups. The event manager could create users without credentials and therefore unable to log in with methods other than OIDC and, optionally, allow them to be added to some groups to inherit their settings. I don't want to have the entire user form in the event manager. You could configure the event manager to add the users to different groups based on username pattern match for example. These users will only be able to log in via OIDC, to enable SFTP they will need to set a password or public key after login. This is something I have in the future ideas of the SFTPGo roadmap but it is just an idea, I need to think about a more concrete plan and ideally it should work as a general replacement for the pre-login hook

[Shaltz]

For future reference, here is a working pre_login_hook that kinda works,
writen in bash for portability and ease of use...

(edited)
check the script below for a fully working version

@drakkan
With this, I am able to create a user on a successfull OIDC connection, everything works,
except for one thing...

The connection cinematic is the following:

1st time the user clicks on "Login with OpenID" (at this point the user doesn't exist in sftpgo)
OIDC login call > OIDC successfull request > call to the pre_login_hook > user creation > user is NOT able to log in and ends up with the error message: "Unable to get the user associated with the OpenID token"

the second time it clicks on "Login with OpenID" it can login just fine (at this point the user exists in sftpgo)

I was expecting something like that :
OIDC login call > OIDC successfull request > call to the pre_login_hook > user creation > user is logged in right after its creation.

Have I missed something somewhere ?

PS: would you be interested that I create a pull request with this script (with some comments added and all) as an example ?
If yes, tell me where you want me to put it ;)

[drakkan]

Sorry to carry on on that subject, but doesn't the POST API call to the users endpoint returns a valid sftpgo user ? I've been trying to echo this value, it's a valid json but I'm still facing the same behaviour as before... :/ when I look at the example that you linked in the first answer, I can't see no difference except for some key/value...

You don't need to use the REST API, just return a valid user as json on the standard output. Check the sftpgo logs to see if there is an error

[Shaltz]

So sorry mate, I didn't get that at all...
thanks so much, after your explanation, everything works perfectly ;)

If anybody gets lost here and is wandering about what script to use,
here is mine

#!/bin/bash
##
##========================================================================================
##                                                                                      ##
##                       SFTPGO: Create a user on OIDC connection                       ##
##                                   with a S3 backend                                  ##
##                                                                                      ##
##========================================================================================
##
## Need to install curl and jq
##
### on debian based system:
### sudo apt install curl jq
##
##──── VARIABLES ─────────────────────────────────────────────────────────────────────────

## S3 related info
S3_BUCKET_NAME="some-bucket"
S3_BUCKET_REGION="us-east-1"
S3_BUCKET_PREFIX="users"

## The template for the user to create
##
## the variable style strings ($username...) are placeholders for jq
## cf: USER_TO_CREATE below
##
### fields are :
##
### status: 0 or 1 (inactive or active)
### username: depending on your OIDC settings, it can be an email address, a surname...
### password (optional): if you want to set a password for this user to connect via FTP/SFTP/WEBDAV...
### permissions: set of permissions for this user (called ACLs in the UI)
### filesystem: 
#### provider: `0` for local filesystem, `1` for S3 backend, `2` for Google Cloud Storage (GCS) backend, `3` for Azure Blob Storage backend, `4` for local encrypted backend, `5` for SFTP backend
#### s3config: config for the S3 backend
SFTPGO_USER_TEMPLATE='{
  "status": 1, 
  "username": $username, 
  "permissions": {
    "/":["*"]
    }, 
    "filesystem": {
      "provider": 1, 
      "s3config": {
        "bucket": $s3_bucket_name,
        "region": $s3_bucket_region, 
        "key_prefix": $s3_prefix
        }
      }
    }'


##──── INTERNAL VARIABLES ────────────────────────────────────────────────────────────────

## The current user ID
##
CURR_USER_ID=$(echo "${SFTPGO_LOGIND_USER}" | jq -r .id)

## The current user name
##
CURR_USER_NAME=$(echo "${SFTPGO_LOGIND_USER}" | jq -r .username)

## The user to create, serialized in JSON
##
USER_TO_CREATE=$(jq --null-input \
--arg username "${CURR_USER_NAME}" \
--arg s3_bucket_name "${S3_BUCKET_NAME}" \
--arg s3_bucket_region "${S3_BUCKET_REGION}" \
--arg s3_prefix "${S3_BUCKET_PREFIX}/${CURR_USER_NAME}" \
"${SFTPGO_USER_TEMPLATE}")

##──── LOGIC ─────────────────────────────────────────────────────────────────────────────

## Is the user coming from an OIDC connection ?
if [[ "${SFTPGO_LOGIND_METHOD,,}" != "idp" || "${SFTPGO_LOGIND_PROTOCOL,,}" != "oidc" ]]
then
  ## The user is not coming from an external IDP or is not connecting via OIDC
    exit 0

## Does the user already exist in SFTPGo ?
### if it's equal to 0, the user DOESN'T exist inside SFTPGo
elif [[ "${CURR_USER_ID}" -ne 0 ]]
then
  # The user already exists in SFTPGo
    exit 0

## Return the user for it to be created automatically
else
    echo "${USER_TO_CREATE}"
    exit 0
fi

Thanks again @drakkan for your help and support ;)

over and out

[drakkan]

So sorry mate, I didn't get that at all... thanks so much, after your explanation, everything works perfectly ;)

If anybody gets lost here and is wandering about what script to use, here is mine

#!/bin/bash
##
##========================================================================================
##                                                                                      ##
##                       SFTPGO: Create a user on OIDC connection                       ##
##                                   with a S3 backend                                  ##
##                                                                                      ##
##========================================================================================
##
## Need to install curl and jq
##
### on debian based system:
### sudo apt install curl jq
##
##──── VARIABLES ─────────────────────────────────────────────────────────────────────────

## S3 related info
S3_BUCKET_NAME="some-bucket"
S3_BUCKET_REGION="us-east-1"
S3_BUCKET_PREFIX="users"

## The template for the user to create
##
## the variable style strings ($username...) are placeholders for jq
## cf: USER_TO_CREATE below
##
### fields are :
##
### status: 0 or 1 (inactive or active)
### username: depending on your OIDC settings, it can be an email address, a surname...
### password (optional): if you want to set a password for this user to connect via FTP/SFTP/WEBDAV...
### permissions: set of permissions for this user (called ACLs in the UI)
### filesystem: 
#### provider: `0` for local filesystem, `1` for S3 backend, `2` for Google Cloud Storage (GCS) backend, `3` for Azure Blob Storage backend, `4` for local encrypted backend, `5` for SFTP backend
#### s3config: config for the S3 backend
SFTPGO_USER_TEMPLATE='{
  "status": 1, 
  "username": $username, 
  "permissions": {
    "/":["*"]
    }, 
    "filesystem": {
      "provider": 1, 
      "s3config": {
        "bucket": $s3_bucket_name,
        "region": $s3_bucket_region, 
        "key_prefix": $s3_prefix
        }
      }
    }'


##──── INTERNAL VARIABLES ────────────────────────────────────────────────────────────────

## The current user ID
##
CURR_USER_ID=$(echo "${SFTPGO_LOGIND_USER}" | jq -r .id)

## The current user name
##
CURR_USER_NAME=$(echo "${SFTPGO_LOGIND_USER}" | jq -r .username)

## The user to create, serialized in JSON
##
USER_TO_CREATE=$(jq --null-input \
--arg username "${CURR_USER_NAME}" \
--arg s3_bucket_name "${S3_BUCKET_NAME}" \
--arg s3_bucket_region "${S3_BUCKET_REGION}" \
--arg s3_prefix "${S3_BUCKET_PREFIX}/${CURR_USER_NAME}" \
"${SFTPGO_USER_TEMPLATE}")

##──── LOGIC ─────────────────────────────────────────────────────────────────────────────

## Is the user coming from an OIDC connection ?
if [[ "${SFTPGO_LOGIND_METHOD,,}" != "idp" || "${SFTPGO_LOGIND_PROTOCOL,,}" != "oidc" ]]
then
  ## The user is not coming from an external IDP or is not connecting via OIDC
    exit 0

## Does the user already exist in SFTPGo ?
### if it's equal to 0, the user DOESN'T exist inside SFTPGo
elif [[ "${CURR_USER_ID}" -ne 0 ]]
then
  # The user already exists in SFTPGo
    exit 0

## Return the user for it to be created automatically
else
    echo "${USER_TO_CREATE}"
    exit 0
fi

Thanks again @drakkan for your help and support ;)

over and out

glad you fixed your script. I hope you are using the AWS offering to help the project, thanks. It's very cheap

[Shaltz]

@drakkan
Will do for sure ;)...
I am actually trying to convince and see with the accounting departement
to become a sponsor. Not sure yet if we could go down that path, but the option
is on the table.
I'll keep you posted ;)

thanks again for all your work :)

[marcorotella]

Dear @drakkan and @Shaltz thanks for sharing this very useful information. I was able to set this up with the provided script and now i am stuck at the next level.

In my test environment i have setup a group and assigned a bunch of virtual folders (S3 buckets) to it. The idea is to have the user logging in via MS Entra SSO, assigned automatically to the group and the virtual folders as a consequence, made available to the user automatically.

I edited the user template as follows and it works, as long as the group is assigned properly to the user.
Unfortunately the virtual folders are not inherited by the user in this way, although assigning the user to the group manually does the job. Am i missing something?

SFTPGO_USER_TEMPLATE='{
  "status": 1, 
  "username": $username, 
  "permissions": {
    "/":["*"]
    }, 
  "groups": [
        {
            "name": "my_group",
            "type": 1
        }
    ]
  }'