Kubernetes
Kubernetes is a container orchestration engine for automating deployment, scaling, and management of containerized applications. This page describes the details of Webswing Cluster in Kubernetes. Following guide provides examples only and actual deployments may vary depending on the requirements and infrastructure provider.
Images
Webswing Cluster Deployment examples rely on Docker images that are stored in our private Docker registry. Access to this registry is limited according to the purchased SLA. Our examples are meant for demonstration purposes only. You might need a different base image OS, Java version, ...
Base Images - Dockerfiles
In case you want to create your own base images from scratch use following Dockerfiles as an example. We are using these to create the base images.
dev.webswing.org/webswing-admin:23.1
webswing-admin-23.1-distribution.zip to be present in the same folder as Dockerfile
run docker build -t dev.webswing.org/webswing-admin:23.1 .
Dockerfile:
FROM openjdk:11-jre
RUN apt-get update && apt-get install --no-install-recommends -y \
wget \
unzip \
less \
nano \
procps \
dos2unix
COPY *.zip /opt/webswing-admin/dist.zip
RUN unzip /opt/webswing-admin/dist.zip -d /opt/webswing-admin && \
mv /opt/webswing-admin/webswing-admin-*/* /opt/webswing-admin && \
rm -d /opt/webswing-admin/dist.zip /opt/webswing-admin/webswing-admin-*/
COPY *.war /opt/webswing-admin/webswing-admin-server.war
ENV WEBSWING_HOME=/opt/webswing-admin
COPY docker/docker.tgz .
RUN tar xzvf docker.tgz --strip 1 -C /usr/local/bin docker/docker && rm docker.tgz
WORKDIR $WEBSWING_HOME
RUN mkdir -p /etc/service/webswing-admin && \
echo "#!/bin/sh\ncd $WEBSWING_HOME\nexec $JAVA_HOME/bin/java \$1 -jar $WEBSWING_HOME/server/webswing-jetty-launcher.jar -admin \$2" > /etc/service/webswing-admin/run && \
chmod +x /etc/service/webswing-admin/run
EXPOSE 8080
CMD /etc/service/webswing-admin/run "$ADMIN_JAVA_OPTS" "$ADMIN_OPTS"
dev.webswing.org/webswing-ce-server:23.1
webswing-cluster-23.1-distribution.zip to be present in the same folder as Dockerfile
run docker build -t dev.webswing.org/webswing-ce-server:23.1 .
Dockerfile:
FROM openjdk:11-jre
RUN apt-get update && apt-get install --no-install-recommends -y \
wget \
unzip \
less \
nano \
procps
COPY *.zip /opt/webswing-cluster/dist.zip
RUN unzip /opt/webswing-cluster/dist.zip -d /opt/webswing-cluster && \
mv /opt/webswing-cluster/webswing-cluster-*/* /opt/webswing-cluster && \
rm -d /opt/webswing-cluster/dist.zip /opt/webswing-cluster/webswing-cluster-*/
COPY *.war /opt/webswing-cluster/webswing.war
ENV WEBSWING_HOME=/opt/webswing-cluster
WORKDIR $WEBSWING_HOME
RUN mkdir -p /etc/service/webswing-cluster && \
echo "#!/bin/sh\ncd $WEBSWING_HOME\nexec $JAVA_HOME/bin/java \$1 -jar $WEBSWING_HOME/server/webswing-jetty-launcher.jar \$2" > /etc/service/webswing-cluster/run && \
chmod +x /etc/service/webswing-cluster/run
EXPOSE 8080
CMD /etc/service/webswing-cluster/run "$WEBSWING_JAVA_OPTS" "$WEBSWING_OPTS"
dev.webswing.org/webswing-ce-session:23.1
webswing-cluster-23.1-distribution.zip to be present in the same folder as Dockerfile
run docker build -t dev.webswing.org/webswing-ce-session:23.1 .
Dockerfile:
FROM openjdk:11-jre
RUN apt-get update && apt-get install --no-install-recommends -y \
wget \
unzip \
xvfb \
libxext6 \
libxi6 \
libxtst6 \
libxrender1 \
libpangoft2-1.0-0 \
dos2unix \
less \
nano \
procps
COPY *.zip /opt/webswing-sessionpool/dist.zip
RUN unzip /opt/webswing-sessionpool/dist.zip -d /opt/webswing-sessionpool && \
mv /opt/webswing-sessionpool/webswing-cluster-*/* /opt/webswing-sessionpool && \
rm -d /opt/webswing-sessionpool/dist.zip /opt/webswing-sessionpool/webswing-cluster-*/
COPY *.war /opt/webswing-sessionpool/webswing.war
ENV WEBSWING_HOME=/opt/webswing-sessionpool \
DISPLAY=:99
WORKDIR $WEBSWING_HOME
RUN mkdir -p /etc/service/xvfb && \
echo "#!/bin/sh\nexec Xvfb :99" > /etc/service/xvfb/run && \
chmod +x /etc/service/xvfb/run
RUN mkdir -p /etc/service/webswing-sessionpool && \
echo "#!/bin/sh\ncd $WEBSWING_HOME\nexec $JAVA_HOME/bin/java \$1 -jar $WEBSWING_HOME/webswing.war \$2" > /etc/service/webswing-sessionpool/run && \
chmod +x /etc/service/webswing-sessionpool/run
COPY start.sh /opt/webswing-sessionpool/start.sh
RUN dos2unix /opt/webswing-sessionpool/start.sh
RUN chmod +x /opt/webswing-sessionpool/start.sh
CMD ./start.sh
In case of start.sh, please ensure LF line endings. CRLF endings will result in file not found. Please note the flag sessionpool.close.with.session=true. In case you do not provide environment variable WEBSWING_JAVA_OPTS it will shut down the session pool with first application that exits. This is useful in case of Pod per session and downscaling.
start.sh:
#!/bin/bash
WS_OPTS="-sessionpool"
WS_JAVA_OPTS="-Xmx128M -Djava.net.preferIPv4Stack=true"
if [ -n "$WEBSWING_OPTS" ]; then
WS_OPTS=$WEBSWING_OPTS
fi
if [ -n "$WEBSWING_JAVA_OPTS" ]; then
WS_JAVA_OPTS=$WEBSWING_JAVA_OPTS
fi
rm -f /tmp/.X99-lock
/etc/service/xvfb/run &
/etc/service/webswing-sessionpool/run "$WS_JAVA_OPTS" "$WS_OPTS"
Configuration
Following examples are extending the base images described above. You can merge the Dockerfiles and create the images you need.
Admin Console
run docker build -t dev.webswing.org/webswing-admin:23.1_demo .
Dockerfile:
FROM dev.webswing.org/webswing-admin:23.1
COPY ./webswing-admin.properties /opt/webswing-admin/webswing-admin.properties
CMD /etc/service/webswing-admin/run "$ADMIN_JAVA_OPTS" "$ADMIN_OPTS"
webswing-admin.properties:
# secret signing key for JWT tokens, should be at least 128 characters long string
# - change this to a random alphanumeric string in production
# - same secret must be present in webswing server's webswing.properties
webswing.connection.secret = k8s_secret_FJDBJFNSKJFNJSDNFFSDBFBDSJNCSJNDJKCHDSFJSDKFSDHJFNSDKJFSDNSDKJNCJSDNKJCNSJDNCJKDBSJFBSDJKBFJKSDBKJFBSDKBFKSDJBFBSDFBK
# public URL on which applications are accessible on public internet
webswing.server.publicUrl = http://localhost:30080
# type of websocket URL loader
# - propertyFile (default) - loads from this property file from webswing.server.websocketUrl property
# - propertyFile_noReload - loads from this property file from webswing.server.websocketUrl property, doesn't attempt to reload the property file after start
# - script - loads from a custom script
webswing.websocketUrlLoader.type = propertyFile
# websocket URL loader reloading interval in seconds
webswing.websocketUrlLoader.interval = 5
# use this for webswing.websocketUrlLoader.type = (propertyFile|propertyFile_noReload)
# comma-separated list of websocket URLs to webswing cluster servers
webswing.server.websocketUrl = ws://webswing-server-0.webswing-server.default.svc.cluster.local:8080,ws://webswing-server-1.webswing-server.default.svc.cluster.local:8080
# use this for webswing.websocketUrlLoader.type = script
# script that loads a comma-separated list of websocket URLs
#webswing.websocketUrlLoader.script = urls.bat
# path where admin console logs should be stored
webswing.logsDir = logs/
# path where webswing-server logs are stored, these logs will show in admin console
webswing.server.logsDir = logs/
Webswing Server
Webswing server needs to contain or reference the application web data as described here: cluster/cluster.html#web-configuration This server example extends the base image by copying the application and some configuration to the base image. It uses following folder structure, and corresponding configuration:
./apps/FlatLaf/web/index.html
./apps/FlatLaf/icon.png
./Dockerfile
./webswing.properties
./webswing-server.config
run docker build -t dev.webswing.org/webswing-ce-server:23.1_demo .
Dockerfile:
FROM dev.webswing.org/webswing-ce-server:23.1
COPY ./webswing.properties /opt/webswing-cluster/webswing.properties
COPY ./webswing-server.config /opt/webswing-cluster/webswing-server.config
COPY ./apps /opt/webswing-cluster/apps
CMD /etc/service/webswing-cluster/run "$WEBSWING_JAVA_OPTS" "$WEBSWING_OPTS"
webswing.properties:
# secret signing key for JWT tokens, should be at least 128 characters long string
# - change this to a random alphanumeric string in production
# - same secret must be present in session pool's webswing-sessionpool.properties or admin console's webswing-admin.properties
webswing.connection.secret = k8s_secret_FJDBJFNSKJFNJSDNFFSDBFBDSJNCSJNDJKCHDSFJSDKFSDHJFNSDKJFSDNSDKJNCJSDNKJCNSJDNCJKDBSJFBSDJKBFJKSDBKJFBSDKBFKSDJBFBSDFBK
# websocket URL of this server, if deployed outside embedded Jetty
#webswing.server.websocketUrl = ws://localhost:8080
webswing.logsDir = logs/
**webswing-server.config:
{
"/" : {
"path" : "/",
"security" : {
"module" : "EMBEDED",
"config" : {
"users" : [ {
"username" : "admin",
"password" : "pwd",
"roles" : [ "admin" ]
}, {
"username" : "support",
"password" : "pwd",
"roles" : [ "support" ]
}, {
"username" : "user",
"password" : "pwd"
} ]
},
"classPath" : [ ]
},
"adminConsoleUrl" : "${webswing.admin.url}",
"langFolder" : "${webswing.configDir}/lang",
"homeDir" : "${user.dir}",
"allowedCorsOrigins" : [ "*" ],
"dataStore" : {
"module" : "FILESYSTEM",
"config" : {
"threadDumpsFolder" : "${WEBSWING_DATA_STORE}/threadDumps",
"recordingsFolder" : "${WEBSWING_DATA_STORE}/recordings",
"transferFolder" : "${WEBSWING_DATA_STORE}/transfer"
}
}
},
"/demo" : {
"path" : "/demo",
"name" : "Demo",
"webFolder" : "${webswing.configDir}/apps/FlatLaf/web",
"security" : {
"module" : "NONE",
"classPath" : [ ],
"config" : { },
"authorizationConfig" : {
"users" : [ ],
"roles" : [ ]
}
},
"icon" : "${webswing.configDir}/apps/FlatLaf/icon.png",
"webHomeDir" : "/tmp",
"langFolder" : "${webswing.configDir}/lang",
"uploadMaxSize" : 5,
"maxClients" : -1,
"sessionMode" : "ALWAYS_NEW_SESSION",
"allowStealSession" : false,
"autoLogout" : false,
"enabled" : true,
"restrictedResources" : [ ],
"monitorEdtEnabled" : true,
"loadingAnimationDelay" : 2,
"dataStore" : {
"module" : "INHERITED",
"config" : { }
},
"recordingConsentRequired" : false,
"mirroringConsentRequired" : false,
"goodbyeUrl" : ""
}
}
Webswing Session Pool - Static cluster
Webswing Session Pool is responsible for launchiong the Java Application. It needs to contain or reference the applications jar files as described here: cluster/cluster.html#app-configuration This sesion pool example extends the base image by copying the application and some configuration to the base image. It uses following folder structure, and corresponding configuration:
./apps/FlatLaf/flatlaf-demo-1.1.2.jar
./Dockerfile
./webswing-session.properties
./webswing-app.config
run docker build -t dev.webswing.org/webswing-ce-session:23.1_demo .
Dockerfile:
FROM dev.webswing.org/webswing-ce-session:23.1
COPY ./webswing-app.config /opt/webswing-sessionpool/webswing-app.config
COPY ./webswing-session.properties /opt/webswing-sessionpool/webswing-sessionpool.properties
COPY ./FlatLaf /opt/webswing-sessionpool/apps/FlatLaf
CMD ./start.sh
webswing-session.properties:
# secret signing key for JWT tokens, should be at least 128 characters long string
# - change this to a random alphanumeric string in production
# - same secret must be present in cluster server's webswing.properties
webswing.connection.secret = k8s_secret_FJDBJFNSKJFNJSDNFFSDBFBDSJNCSJNDJKCHDSFJSDKFSDHJFNSDKJFSDNSDKJNCJSDNKJCNSJDNCJKDBSJFBSDJKBFJKSDBKJFBSDKBFKSDJBFBSDFBK
# session pool id is generated if not provided
#sessionpool.id = session-pool-0
# higher number means higher priority, you can use this value in load balancer algorithm
sessionpool.priority = 1
# maximum pool size, set -1 for infinite
sessionpool.instances.max = 10
# reconnect policy (omit to disable reconnecting, session pool will shutdown with no active ws connection)
# - use interval, retries or both
# - session pool will try to reconnect as many as webswing.server.reconnect.retries times
# - session pool will try to reconnect for the duration of webswing.server.reconnect.interval interval
# reconnect interval - try to reconnect on connection loss for given amount of seconds
# - use -1 for infinite interval
sessionpool.reconnect.interval = -1
# reconnect retries - try to reconnect on connection loss for given amount of seconds
#sessionpool.reconnect.retries = 5
# delay between reconnects, in milliseconds
sessionpool.reconnect.delay = 3000
# type of websocket URL loader
# - propertyFile (default) - loads from this property file from webswing.server.websocketUrl property
# - propertyFile_noReload - loads from this property file from webswing.server.websocketUrl property, doesn't attempt to reload the property file after start
# - script - loads from a custom script
webswing.websocketUrlLoader.type = propertyFile
# websocket URL loader reloading interval in seconds
webswing.websocketUrlLoader.interval = 5
# use this for webswing.websocketUrlLoader.type = (propertyFile|propertyFile_noReload)
# comma-separated list of websocket URLs to webswing cluster servers
webswing.server.websocketUrl = ws://webswing-server-0.webswing-server.default.svc.cluster.local:8080,ws://webswing-server-1.webswing-server.default.svc.cluster.local:8080
# use this for webswing.websocketUrlLoader.type = script
# script that loads a comma-separated list of websocket URLs
#webswing.websocketUrlLoader.script = urls.bat
webswing.logsDir = logs/
webswing-app.config:
{
"/demo" : {
"jreExecutable" : "${java.home}/bin/java",
"javaVersion" : "${java.version}",
"vmArgs" : "-Xmx128m -DauthorizedUser=${user}",
"classPathEntries" : [ "/opt/webswing-sessionpool/apps/FlatLaf/*.jar" ],
"theme" : "Murrine",
"fontConfig" : { },
"swingSessionTimeout" : 60,
"isolatedFs" : true,
"debug" : true,
"javaFx" : false,
"javaFxClassPathEntries" : [ ],
"directdraw" : true,
"compositingWinManager" : true,
"allowDelete" : true,
"allowDownload" : true,
"allowAutoDownload" : true,
"allowUpload" : true,
"allowJsLink" : true,
"jsLinkWhiteList" : [ "org.webswing.demo.jslink.JsLinkDemo.*" ],
"launcherType" : "Desktop",
"launcherConfig" : {
"mainClass" : "com.formdev.flatlaf.demo.FlatLafDemo"
},
"userDir" : "${user}",
"homeDir" : "/opt/webswing-sessionpool/apps/FlatLaf",
"transparentFileSave" : true,
"clearTransferDir" : true,
"transparentFileOpen" : true,
"transferDir" : "${user}/upload",
"timeoutIfInactive" : false,
"sessionLogging" : false,
"jsLinkWhitelist" : [ "*" ],
"allowLocalClipboard" : true,
"allowServerPrinting" : false,
"dockMode" : "ALL",
"allowStatisticsLogging" : true,
"testMode" : false
}
}
Webswing Session Pool - Autoscaling Experimental
For the autoscaling we need to change the maximum number of the instances per session pool to 1. All the rest remains the same. The difference is in starting the session pools which is handled by a scaling algorithm.
webswing-session.properties
sessionpool.instances.max = 1
Webswing Scaler - Autoscaling Experimental
Webswing scaler is an example on how to manage Kubernetes pods and sessions. Due to statefulness of the JVM application and different longevity of the sessions, each session is served by a dedicated POD. Once the application finishes the POD can be terminated. Some free PODs are desirable in order to ensure quick start and good user experience.
run docker build -t dev.webswing.org/webswing-scaler:23.1_demo .
Dockerfile:
FROM python:3.6-alpine
RUN pip install kubernetes
WORKDIR /app
COPY scaler.py scaler.py
CMD [ "python", "/app/scaler.py"]
Deployment
Admin Console
Admin Console is configured mainly by the webswing-admin.properties that is provided in the image.
webswing-admin.yaml
kind: Service
apiVersion: v1
metadata:
name: webswing-admin
spec:
type: NodePort
ports:
- port: 8090
targetPort: 8090
nodePort: 30090
selector:
app: webswing-admin
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: webswing-admin
spec:
replicas: 1
selector:
matchLabels:
app: webswing-admin
template:
metadata:
labels:
app: webswing-admin
spec:
containers:
- name: webswing-admin
image: dev.webswing.org/webswing-admin:23.1_demo
imagePullPolicy: IfNotPresent
ports:
- name: web
containerPort: 8090
- name: ws
containerPort: 8000
env:
- name: ADMIN_JAVA_OPTS
value: "-Dwebswing.server.publicUrl=http://localhost:30080"
env:
- name: ADMIN_OPTS
value: "-h 0.0.0.0 -j /opt/webswing-admin/jetty.properties"
Server
webswing-server.yaml:
kind: Service
apiVersion: v1
metadata:
name: webswing-server
spec:
type: NodePort
ports:
- port: 8080
targetPort: 8080
nodePort: 30080
selector:
app: webswing-server
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: webswing-server
spec:
replicas: 2
selector:
matchLabels:
app: webswing-server
serviceName: "webswing-server"
template:
metadata:
labels:
app: webswing-server
spec:
containers:
- name: webswing-server
image: dev.webswing.org/webswing-ce-server:23.1_demo
imagePullPolicy: IfNotPresent
ports:
- name: web
containerPort: 8080
env:
- name: WEBSWING_JAVA_OPTS
value: "-Xmx256M -Djava.net.preferIPv4Stack=true -Dwebswing.admin.url=http://localhost:30090"
- name: WEBSWING_OPTS
value: "-h 0.0.0.0 -j /opt/webswing-cluster/jetty.properties"
Session Pool - Static Session Pools
webswing-sessionpool.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: webswing-sessionpool
spec:
replicas: 2
selector:
matchLabels:
app: webswing-sessionpool
serviceName: "webswing-sessionpool"
template:
metadata:
labels:
app: webswing-sessionpool
spec:
containers:
- name: webswing-sessionpool
image: dev.webswing.org/webswing-ce-session:23.1_demo
imagePullPolicy: IfNotPresent
ports:
- name: web
containerPort: 8080
- name: ws
containerPort: 8000
env:
- name: WEBSWING_JAVA_OPTS
value: "-Xmx128M -Djava.net.preferIPv4Stack=true"
Scaler - Dynamic Session Creation
Scaling feature has been reimplemented in the Admin Console. Following part is for reference only.
In case of kubernetes example with autoscaling the Session lifecycle is managed by the scaling script. In the scaler deployment descriptor we tell the scaler what is the desired buffer (MIN_FREE_PODS) and what session image to create.
webswing-scaler.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: webswing-scaler
spec:
selector:
matchLabels:
app: webswing-scaler
template:
metadata:
labels:
app: webswing-scaler
spec:
containers:
- name: webswing-scaler
image: dev.webswing.org/webswing-ce-scaler:23.1_demo
imagePullPolicy: IfNotPresent
env:
- name: PYTHONUNBUFFERED
value: "0"
- name: WEBSWING_IMAGE
value: "dev.webswing.org/webswing-ce-session:23.1_demo"
- name: MIN_FREE_PODS
value: "2"
- name: MAX_PENDING_PODS
value: "2"