Metadata-Version: 2.4
Name: zndraw
Version: 0.6.0a7
Summary: ZnDraw
License: Eclipse Public License - v 2.0
        
            THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE
            PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION
            OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
        
        1. DEFINITIONS
        
        "Contribution" means:
        
          a) in the case of the initial Contributor, the initial content
             Distributed under this Agreement, and
        
          b) in the case of each subsequent Contributor:
             i) changes to the Program, and
             ii) additions to the Program;
          where such changes and/or additions to the Program originate from
          and are Distributed by that particular Contributor. A Contribution
          "originates" from a Contributor if it was added to the Program by
          such Contributor itself or anyone acting on such Contributor's behalf.
          Contributions do not include changes or additions to the Program that
          are not Modified Works.
        
        "Contributor" means any person or entity that Distributes the Program.
        
        "Licensed Patents" mean patent claims licensable by a Contributor which
        are necessarily infringed by the use or sale of its Contribution alone
        or when combined with the Program.
        
        "Program" means the Contributions Distributed in accordance with this
        Agreement.
        
        "Recipient" means anyone who receives the Program under this Agreement
        or any Secondary License (as applicable), including Contributors.
        
        "Derivative Works" shall mean any work, whether in Source Code or other
        form, that is based on (or derived from) the Program and for which the
        editorial revisions, annotations, elaborations, or other modifications
        represent, as a whole, an original work of authorship.
        
        "Modified Works" shall mean any work in Source Code or other form that
        results from an addition to, deletion from, or modification of the
        contents of the Program, including, for purposes of clarity any new file
        in Source Code form that contains any contents of the Program. Modified
        Works shall not include works that contain only declarations,
        interfaces, types, classes, structures, or files of the Program solely
        in each case in order to link to, bind by name, or subclass the Program
        or Modified Works thereof.
        
        "Distribute" means the acts of a) distributing or b) making available
        in any manner that enables the transfer of a copy.
        
        "Source Code" means the form of a Program preferred for making
        modifications, including but not limited to software source code,
        documentation source, and configuration files.
        
        "Secondary License" means either the GNU General Public License,
        Version 2.0, or any later versions of that license, including any
        exceptions or additional permissions as identified by the initial
        Contributor.
        
        2. GRANT OF RIGHTS
        
          a) Subject to the terms of this Agreement, each Contributor hereby
          grants Recipient a non-exclusive, worldwide, royalty-free copyright
          license to reproduce, prepare Derivative Works of, publicly display,
          publicly perform, Distribute and sublicense the Contribution of such
          Contributor, if any, and such Derivative Works.
        
          b) Subject to the terms of this Agreement, each Contributor hereby
          grants Recipient a non-exclusive, worldwide, royalty-free patent
          license under Licensed Patents to make, use, sell, offer to sell,
          import and otherwise transfer the Contribution of such Contributor,
          if any, in Source Code or other form. This patent license shall
          apply to the combination of the Contribution and the Program if, at
          the time the Contribution is added by the Contributor, such addition
          of the Contribution causes such combination to be covered by the
          Licensed Patents. The patent license shall not apply to any other
          combinations which include the Contribution. No hardware per se is
          licensed hereunder.
        
          c) Recipient understands that although each Contributor grants the
          licenses to its Contributions set forth herein, no assurances are
          provided by any Contributor that the Program does not infringe the
          patent or other intellectual property rights of any other entity.
          Each Contributor disclaims any liability to Recipient for claims
          brought by any other entity based on infringement of intellectual
          property rights or otherwise. As a condition to exercising the
          rights and licenses granted hereunder, each Recipient hereby
          assumes sole responsibility to secure any other intellectual
          property rights needed, if any. For example, if a third party
          patent license is required to allow Recipient to Distribute the
          Program, it is Recipient's responsibility to acquire that license
          before distributing the Program.
        
          d) Each Contributor represents that to its knowledge it has
          sufficient copyright rights in its Contribution, if any, to grant
          the copyright license set forth in this Agreement.
        
          e) Notwithstanding the terms of any Secondary License, no
          Contributor makes additional grants to any Recipient (other than
          those set forth in this Agreement) as a result of such Recipient's
          receipt of the Program under the terms of a Secondary License
          (if permitted under the terms of Section 3).
        
        3. REQUIREMENTS
        
        3.1 If a Contributor Distributes the Program in any form, then:
        
          a) the Program must also be made available as Source Code, in
          accordance with section 3.2, and the Contributor must accompany
          the Program with a statement that the Source Code for the Program
          is available under this Agreement, and informs Recipients how to
          obtain it in a reasonable manner on or through a medium customarily
          used for software exchange; and
        
          b) the Contributor may Distribute the Program under a license
          different than this Agreement, provided that such license:
             i) effectively disclaims on behalf of all other Contributors all
             warranties and conditions, express and implied, including
             warranties or conditions of title and non-infringement, and
             implied warranties or conditions of merchantability and fitness
             for a particular purpose;
        
             ii) effectively excludes on behalf of all other Contributors all
             liability for damages, including direct, indirect, special,
             incidental and consequential damages, such as lost profits;
        
             iii) does not attempt to limit or alter the recipients' rights
             in the Source Code under section 3.2; and
        
             iv) requires any subsequent distribution of the Program by any
             party to be under a license that satisfies the requirements
             of this section 3.
        
        3.2 When the Program is Distributed as Source Code:
        
          a) it must be made available under this Agreement, or if the
          Program (i) is combined with other material in a separate file or
          files made available under a Secondary License, and (ii) the initial
          Contributor attached to the Source Code the notice described in
          Exhibit A of this Agreement, then the Program may be made available
          under the terms of such Secondary Licenses, and
        
          b) a copy of this Agreement must be included with each copy of
          the Program.
        
        3.3 Contributors may not remove or alter any copyright, patent,
        trademark, attribution notices, disclaimers of warranty, or limitations
        of liability ("notices") contained within the Program from any copy of
        the Program which they Distribute, provided that Contributors may add
        their own appropriate notices.
        
        4. COMMERCIAL DISTRIBUTION
        
        Commercial distributors of software may accept certain responsibilities
        with respect to end users, business partners and the like. While this
        license is intended to facilitate the commercial use of the Program,
        the Contributor who includes the Program in a commercial product
        offering should do so in a manner which does not create potential
        liability for other Contributors. Therefore, if a Contributor includes
        the Program in a commercial product offering, such Contributor
        ("Commercial Contributor") hereby agrees to defend and indemnify every
        other Contributor ("Indemnified Contributor") against any losses,
        damages and costs (collectively "Losses") arising from claims, lawsuits
        and other legal actions brought by a third party against the Indemnified
        Contributor to the extent caused by the acts or omissions of such
        Commercial Contributor in connection with its distribution of the Program
        in a commercial product offering. The obligations in this section do not
        apply to any claims or Losses relating to any actual or alleged
        intellectual property infringement. In order to qualify, an Indemnified
        Contributor must: a) promptly notify the Commercial Contributor in
        writing of such claim, and b) allow the Commercial Contributor to control,
        and cooperate with the Commercial Contributor in, the defense and any
        related settlement negotiations. The Indemnified Contributor may
        participate in any such claim at its own expense.
        
        For example, a Contributor might include the Program in a commercial
        product offering, Product X. That Contributor is then a Commercial
        Contributor. If that Commercial Contributor then makes performance
        claims, or offers warranties related to Product X, those performance
        claims and warranties are such Commercial Contributor's responsibility
        alone. Under this section, the Commercial Contributor would have to
        defend claims against the other Contributors related to those performance
        claims and warranties, and if a court requires any other Contributor to
        pay any damages as a result, the Commercial Contributor must pay
        those damages.
        
        5. NO WARRANTY
        
        EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT
        PERMITTED BY APPLICABLE LAW, THE PROGRAM IS PROVIDED ON AN "AS IS"
        BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR
        IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF
        TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR
        PURPOSE. Each Recipient is solely responsible for determining the
        appropriateness of using and distributing the Program and assumes all
        risks associated with its exercise of rights under this Agreement,
        including but not limited to the risks and costs of program errors,
        compliance with applicable laws, damage to or loss of data, programs
        or equipment, and unavailability or interruption of operations.
        
        6. DISCLAIMER OF LIABILITY
        
        EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT
        PERMITTED BY APPLICABLE LAW, NEITHER RECIPIENT NOR ANY CONTRIBUTORS
        SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
        EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST
        PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
        CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
        ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE
        EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE
        POSSIBILITY OF SUCH DAMAGES.
        
        7. GENERAL
        
        If any provision of this Agreement is invalid or unenforceable under
        applicable law, it shall not affect the validity or enforceability of
        the remainder of the terms of this Agreement, and without further
        action by the parties hereto, such provision shall be reformed to the
        minimum extent necessary to make such provision valid and enforceable.
        
        If Recipient institutes patent litigation against any entity
        (including a cross-claim or counterclaim in a lawsuit) alleging that the
        Program itself (excluding combinations of the Program with other software
        or hardware) infringes such Recipient's patent(s), then such Recipient's
        rights granted under Section 2(b) shall terminate as of the date such
        litigation is filed.
        
        All Recipient's rights under this Agreement shall terminate if it
        fails to comply with any of the material terms or conditions of this
        Agreement and does not cure such failure in a reasonable period of
        time after becoming aware of such noncompliance. If all Recipient's
        rights under this Agreement terminate, Recipient agrees to cease use
        and distribution of the Program as soon as reasonably practicable.
        However, Recipient's obligations under this Agreement and any licenses
        granted by Recipient relating to the Program shall continue and survive.
        
        Everyone is permitted to copy and distribute copies of this Agreement,
        but in order to avoid inconsistency the Agreement is copyrighted and
        may only be modified in the following manner. The Agreement Steward
        reserves the right to publish new versions (including revisions) of
        this Agreement from time to time. No one other than the Agreement
        Steward has the right to modify this Agreement. The Eclipse Foundation
        is the initial Agreement Steward. The Eclipse Foundation may assign the
        responsibility to serve as the Agreement Steward to a suitable separate
        entity. Each new version of the Agreement will be given a distinguishing
        version number. The Program (including Contributions) may always be
        Distributed subject to the version of the Agreement under which it was
        received. In addition, after a new version of the Agreement is published,
        Contributor may elect to Distribute the Program (including its
        Contributions) under the new version.
        
        Except as expressly stated in Sections 2(a) and 2(b) above, Recipient
        receives no rights or licenses to the intellectual property of any
        Contributor under this Agreement, whether expressly, by implication,
        estoppel or otherwise. All rights in the Program not expressly granted
        under this Agreement are reserved. Nothing in this Agreement is intended
        to be enforceable by any entity that is not a Contributor or Recipient.
        No third-party beneficiary rights are created under this Agreement.
        
        Exhibit A - Form of Secondary Licenses Notice
        
        "This Source Code may also be made available under the following
        Secondary Licenses when the conditions for such availability set forth
        in the Eclipse Public License, v. 2.0 are satisfied: {name license(s),
        version(s), and exceptions or additional permissions here}."
        
          Simply including a copy of this Agreement, including this Exhibit A
          is not sufficient to license the Source Code under Secondary Licenses.
        
          If it is not possible or desirable to put the notice in a particular
          file, then You may include the notice in a location (such as a LICENSE
          file in a relevant directory) where a recipient would be likely to
          look for such a notice.
        
          You may add additional accurate notices of copyright ownership.
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: cachetools>=6.2.0
Requires-Dist: celery>=5.5.3
Requires-Dist: eventlet>=0.40.3
Requires-Dist: flask-socketio>=5.5.1
Requires-Dist: flask>=3.1.2
Requires-Dist: msgpack>=1.1.1
Requires-Dist: networkx>=3.5
Requires-Dist: pandas>=2.3.3
Requires-Dist: plotly>=6.3.1
Requires-Dist: pydantic>=2.11.9
Requires-Dist: pyjwt>=2.10.1
Requires-Dist: python-socketio[client]>=5.13.0
Requires-Dist: rdkit2ase>=0.1.13
Requires-Dist: redis>=6.4.0
Requires-Dist: tqdm>=4.67.1
Requires-Dist: typer>=0.19.2
Requires-Dist: vesin>=0.3.8
Requires-Dist: zarr>=3.1.3
Requires-Dist: znh5md>=0.4.8
Requires-Dist: znsocket>=0.2.13a3
Provides-Extra: test
Requires-Dist: pytest>=8.4.2; extra == 'test'
Description-Content-Type: text/markdown

1. run the server `uv run src/server.py`

# Goal
Split communication into data channels and control channels.
- Data channels: use HTTP PUT/POST to upload/download data. Use query strings to select what should be loaded, e.g. `?data=positions,species` or just`?data=energy`.
- Control channels: use Socket.IO and Redis for state management and locks

Optional: Control logic, the server knows the previous and the next frame and can check what needs to be updated, e.g. only positions or also species, box, ...

Data persistence should be abstracted away via a `DataProvider` interface.
- store e.g. as npy files for fast access once accessed once. (Possibility to hash??)

Data Edits
- not only store the frame id but also the version `{ "frame": 10, "version": 123 }` would also allow undo operations, (also conflict detection -> considering the edit was made on version 1 but the current version is 3, the edit cannot be applied)
- use / broadcast a lock mechanism (redis lock?) e.g. if the data is to be modified we need a lock, data can only be updated if the lock is held by the client that wants to update it (server-side check). Per-frame lock or even more granular? Timeouts: what if a client crashes mid-edit, socketio disconnect event should trigger? Renewals: if an edit takes long, client must refresh lock before expiry. Server authority: server must check that only the lock-holder can commit.
- with `vis.lock` ... ? and `vis.extend` will check if the lock has been aquired, otherwise aquire it.
- To add / remove and to edit, there exist two entries, one for trajectory indices and another one for metadata per frame. To insert at index N, you must first move the last frame (M) to M+1, then M-1 to M, and so on, until you move frame N to N+1.
```
# 1. Trajectory-Level: A single key holding the list of all frames.
"trajectory:indices"  (Sorted Set)
  -> [ 0, 1, 3, 4, ... ]

# 2. Frame-Level: A separate key for each frame's metadata.
"metadata:frame:0"    (Hash)
  -> { "version": 5, "hash": "..." }

"metadata:frame:1"    (Hash)
  -> { "version": 2, "hash": "..." }

"metadata:frame:3"    (Hash)
  -> { "version": 8, "hash": "..." }
```

Rooms
- use `from flask_socketio import join_room, leave_room` and `
```javascript
socket.on('connect', () => {
  socket.emit('join_room', { room: room_id });
});
```
and use keys like `room:project-alpha:lock:frame:10` and `/data/<room>/...`

For playback, we use the `"Presenter Token" Hybrid Model` approach.
1. Acquire Token: When a user starts scrubbing, their client first asks the server for the "presenter token."
socket.emit('request_presenter_token')

2. Server Grants Token: The server uses SETNX in Redis to grant the token to the first client who asks. This token has a short expiry (e.g., 5 seconds).
r.set("room:project-alpha:presenter_lock", client_id, nx=True, ex=5)

3. Stream Updates: If the client gets the token, it then starts sending the stream of set_room_frame messages just like in your model. The server only accepts these messages from the client who currently holds the token.

4. Renew Token: As long as the user is actively scrubbing, the client sends a renew_presenter_token message every few seconds to extend the expiry.

5. Release/Expire: When the user stops scrubbing, their client explicitly sends a release_presenter_token message. If they close the tab or crash, the token automatically expires in Redis after 5 seconds, allowing someone else to take control.

Conflict detection
- Server only accepts update if client’s base version == current version, If mismatch → reject with "conflict", client must rebase on latest version.

Data hashing
- Detect corruption / incomplete transfer.
- Enable deduplication (same frame content doesn’t need to be re-saved).
- store alongside in db `{ "frame": 10, "version": 123, "hash": "sha256:abcd..." }`

Split into room and client-ids?

Use https://msgpack.org/

How to handle `default` room
Store data in `zarr` format. Use padding for variable length data.

When creating a new room, one should be able to specify the base data.


## Summary
✅ Things that are already great
	•	Split channels
	•	Data over HTTP (upload/download, efficient for MB payloads).
	•	Control over Socket.IO + Redis (events, synchronization, locks).
	•	This avoids mixing heavy + light traffic.
	•	DataProvider abstraction
	•	Hides persistence details (HDF5, .npy, object store, …).
	•	Makes it possible to swap implementations without touching business logic.
	•	You can even wrap in-memory cache behind the interface.
	•	Versioning ({frame, version})
	•	Enables undo/redo and conflict detection (if a client edits an outdated version).
	•	Very good idea for collaborative edits.
	•	Locks (Redis or server-side)
	•	Prevents race conditions (two clients trying to edit same frame).
	•	Redis locks scale well if you have multiple server processes.

⸻

🔹 Improvements & Missing Pieces

1. Lock granularity
	•	You probably want per-frame locks, not global locks.
	•	i.e. lock:frame:10 instead of lock:all.
	•	This way two clients can edit different frames simultaneously.
	•	Use Redis SETNX with expiry for robust distributed locks:

r.set(f"lock:frame:{frame_id}", client_id, nx=True, ex=10)



2. Lock lifecycle
	•	You’ll need a strategy for:
	•	Timeouts: what if a client crashes mid-edit?
	•	Renewals: if an edit takes long, client must refresh lock before expiry.
	•	Server authority: server must check that only the lock-holder can commit.

👉 The Redis community has patterns here (Redlock algorithm).

⸻

3. Conflict detection
	•	Versioning is great, but you should enforce:
	•	Server only accepts update if client’s base version == current version.
	•	If not, reject or trigger conflict resolution.

client -> { "frame": 10, "base_version": 123, "new_data": ... }
server -> compares base_version with stored version (123 vs 124)

	•	If mismatch → reject with "conflict", client must rebase on latest version.

⸻

4. Data hashing
	•	Adding hashes has two benefits:
	•	Detect corruption / incomplete transfer.
	•	Enable deduplication (same frame content doesn’t need to be re-saved).
	•	You can store hash in Redis along with version:

{ "frame": 10, "version": 123, "hash": "sha256:abcd..." }



⸻

5. Persistence backend strategy
	•	.npy is great for fast local random access.
	•	For large-scale / multi-machine setup:
	•	Object storage (S3, MinIO) with pre-signed URLs for uploads/downloads.
	•	Zarr/TileDB for chunked array storage.
	•	Your DataProvider abstraction should make it easy to swap between these.

⸻

6. Partial updates (optional optimization)
	•	If edits are often small (e.g. just moving atoms), sending the entire frame blob is overkill.
	•	You could allow patches (delta updates).
	•	Control channel:

{ "frame": 10, "base_version": 123, "delta": { "positions": [...] } }


	•	Server applies delta → increments version → broadcasts update.

Not critical to start with, but worth keeping in mind.

⸻

7. Broadcast strategy
	•	When a frame is updated:
	•	Don’t broadcast the heavy payload.
	•	Only broadcast metadata { "frame": 10, "version": 124 }.
	•	Clients then fetch updated frame data via HTTP if they care about it.

This keeps the control channel lightweight.

⸻

🔹 Refined workflow (with your ideas included)
	1.	Client requests lock
	•	socket.io.emit("lock_request", {frame: 10})
	•	Server acquires Redis lock → grants if free.
	2.	Client uploads data
	•	HTTP POST /upload/frame/10?lock_token=xyz with new blob.
	3.	Server checks & commits
	•	Verifies lock holder.
	•	Verifies base_version matches current version.
	•	Writes new data (DataProvider.save).
	•	Increments version, stores hash.
	•	Releases lock.
	4.	Server broadcasts update
	•	socket.io.emit("frame_updated", {frame: 10, version: 124}).
	5.	Other clients fetch new data
	•	On receiving "frame_updated", request updated frame from data channel.

⸻

✅ This gives you:
	•	Fast uploads/downloads (HTTP).
	•	Strong synchronization (Redis locks + versions).
	•	Robust persistence (swappable backends).
	•	Undo/redo potential (versions + hashes).
	•	No race conditions (lock + version checks).


TODOs
- have the state of the queue available in the chat or a table or something