UESPWiki - User Contributions [en]

UESPWiki:Servers

2026-05-06T14:43:08Z

Daveh:

{{Trail}}
The basic server architecture is illustrated in the following schematic (updated May 2026):
[[File:UESP-stats-Servers.png|center]]

The following table lists all primary and backup '''servers''' used by the [[UESPWiki:About|UESP]] (updated in May 2026).
<center>
{| class="wikitable"
|-
! Server !! Purpose !! Specs
|-
| '''N/A''' || Firewall||
* Managed firewall from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* FortiGate FG-100F Firewall
* Servers sit behind this device within a private network
* 1 Gbps WAN, 30TB/Month
* 1 Gbps LAN, 30TB/Month
|-
| '''content1''' || Wiki Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Apache 2.4, PHP 7.4+
|-
| '''content2''' || Wiki Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Apache 2.4, PHP 7.4+
|-
| '''content3''' || Wiki Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Apache 2.4, PHP 7.4+
|-
| '''dev1''' || Misc Content, Utility, Dev ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Apache 2.4, PHP 7.4+
|-
| '''db1''' || Write Database ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* AMD EPYC 7413 (24 Cores, 96 Threads, 2.7GHz), 128GB RAM
* (x2) 4TB SSD in software RAID 1
* AlmaLinux 9, MariaDB 10.5
|-
| '''db2''' || Read Database ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* AMD EPYC 7413 (24 Cores, 96 Threads, 2.7GHz), 128GB RAM
* (x2) 4TB SSD in software RAID 1
* AlmaLinux 9, MariaDB 10.5
|-
| '''cache1''' || Cache ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Varnish 6.6, Nginx 1.20
|-
| '''files1''' || Static Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 4TB SSD in software RAID 1
* AlmaLinux 9, Lighttpd 1.4
|-
| '''files2''' || Static Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E5-2620v3 2.4GHz (6 Cores, 24 Threads, 2.4GHz), 64GB RAM
* (x2) 2TB SSD in hardware RAID 1
* CentOS 7, Lighttpd 1.4
|-
| '''search1''' || Static Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Elasticsearch 5+
|-
| '''backup1''' || On-Site Backup ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2278G (8 Cores, 16 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* (x2) 8TB HDD in software RAID 1
* AlmaLinux 9
|-
| '''backup2''' || Off-Site Backup ||
* Custom Build From Parts, Fanless/Quiet Design
* AMD Ryzen 5 5600G (5 Cores, 12 Threads, 4.4GHz), 16GB RAM
* 1TB SSD
* 8TB, 4TB, 4TB HDDs
* CentOS 8
* Home Cable connection (~1000 Mbps download, ~100 Mbps upload)
* 1000W UPS
* Personally hosted, offsite backup
|}
</center>

==Server Access==
The following users have [[UESPWiki:Server Access|server access]]:
{| class=wikitable
! User !! Access Level
|-
| [[User:Daveh|Daveh]] || Full
|-
| [[User:Eckserah|Eckserah]] || Full
|-
| [[User:RobinHood70|RobinHood70]] || Partial (As Needed)
|-
! colspan=2 | Inactive
|}

UESPWiki:Servers

2026-05-06T14:42:42Z

Daveh:

{{Trail}}
The following table lists all primary and backup '''servers''' used by the [[UESPWiki:About|UESP]] (updated in May 2024).

<center>
{| class="wikitable"
|-
! Server !! Purpose !! Specs
|-
| '''N/A''' || Firewall||
* Managed firewall from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* FortiGate FG-100F Firewall
* Servers sit behind this device within a private network
* 1 Gbps WAN, 30TB/Month
* 1 Gbps LAN, 30TB/Month
|-
| '''content1''' || Wiki Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Apache 2.4, PHP 7.4+
|-
| '''content2''' || Wiki Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Apache 2.4, PHP 7.4+
|-
| '''content3''' || Wiki Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Apache 2.4, PHP 7.4+
|-
| '''dev1''' || Misc Content, Utility, Dev ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Apache 2.4, PHP 7.4+
|-
| '''db1''' || Write Database ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* AMD EPYC 7413 (24 Cores, 96 Threads, 2.7GHz), 128GB RAM
* (x2) 4TB SSD in software RAID 1
* AlmaLinux 9, MariaDB 10.5
|-
| '''db2''' || Read Database ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* AMD EPYC 7413 (24 Cores, 96 Threads, 2.7GHz), 128GB RAM
* (x2) 4TB SSD in software RAID 1
* AlmaLinux 9, MariaDB 10.5
|-
| '''cache1''' || Cache ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Varnish 6.6, Nginx 1.20
|-
| '''files1''' || Static Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 4TB SSD in software RAID 1
* AlmaLinux 9, Lighttpd 1.4
|-
| '''files2''' || Static Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E5-2620v3 2.4GHz (6 Cores, 24 Threads, 2.4GHz), 64GB RAM
* (x2) 2TB SSD in hardware RAID 1
* CentOS 7, Lighttpd 1.4
|-
| '''search1''' || Static Content ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2434G (4 Cores, 8 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* AlmaLinux 9, Elasticsearch 5+
|-
| '''backup1''' || On-Site Backup ||
* Dedicated server from [http://leaseweb.com Leaseweb] in Montreal, Quebec
* Intel Xeon E-2278G (8 Cores, 16 Threads, 3.4GHz), 64GB RAM
* (x2) 2TB SSD in software RAID 1
* (x2) 8TB HDD in software RAID 1
* AlmaLinux 9
|-
| '''backup2''' || Off-Site Backup ||
* Custom Build From Parts, Fanless/Quiet Design
* AMD Ryzen 5 5600G (5 Cores, 12 Threads, 4.4GHz), 16GB RAM
* 1TB SSD
* 8TB, 4TB, 4TB HDDs
* CentOS 8
* Home Cable connection (~1000 Mbps download, ~100 Mbps upload)
* 1000W UPS
* Personally hosted, offsite backup
|}
</center>

The basic server architecture is illustrated in the following schematic (updated May 2026):
[[File:UESP-stats-Servers.png|center]]

==Server Access==
The following users have [[UESPWiki:Server Access|server access]]:
{| class=wikitable
! User !! Access Level
|-
| [[User:Daveh|Daveh]] || Full
|-
| [[User:Eckserah|Eckserah]] || Full
|-
| [[User:RobinHood70|RobinHood70]] || Partial (As Needed)
|-
! colspan=2 | Inactive
|}

File:UESP-stats-Servers.png

2026-05-06T14:15:35Z

Daveh: Daveh uploaded a new version of File:UESP-stats-Servers.png

== Summary ==
Schematic of the UESP server architecture.
== Licensing ==
{{Cc-by-sa-2.5}}
[[Category:UESPWiki-Stats Images]]

User:Daveh/ESO Update

2026-05-01T14:07:32Z

Daveh: /* Update Maps */

This page details the procedure of updating the UESP data after a major ESO update/release.

== General Prep and Notes ==
:* Run ESO patcher to get latest version.
:* (Optional) Edit ''content3:/home/uesp/esolog/esolog.index'' and increment/note the log index. Optional but useful if you need to find a specific entry in the new logs.
:* (Optional) Backup/delete the ''uespLog_backupData.txt'' file created by uespLogMonitor.
:* PTS/Live updates are slightly different. Most steps are for both updates while some steps are for only one or the other.
:* Note that the ''VERSION'' field used for scripts/databases is just the update number itself or the update number with pts appended to it. For example:
::* Live: 17
::* PTS: 17pts

== Special Notes ==
Things that need manually fixing for each update:
:* Fix the Undaunted Command stamina coefficients:
UPDATE uesp_esolog.skillTooltips SET coefType=32, a=0.02, b=0, c=0, r=1 WHERE abilityId=55584 AND idx=1;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.02, b=0, c=0, r=1 WHERE abilityId=55584 AND idx=2;
UPDATE uesp_esolog.skillTooltips SET coefType=1, a=0.02, b=0, c=0, r=1 WHERE abilityId=55584 AND idx=3;
UPDATE uesp_esolog.skillTooltips SET coefType=32, a=0.04, b=0, c=0, r=1 WHERE abilityId=55676 AND idx=1;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.04, b=0, c=0, r=1 WHERE abilityId=55676 AND idx=2;
UPDATE uesp_esolog.skillTooltips SET coefType=1, a=0.04, b=0, c=0, r=1 WHERE abilityId=55676 AND idx=3;
UPDATE uesp_esolog.minedSkills SET a1=0.02, b1=0, c1=0, type2=4, a2=0.02, b2=0, c2=0, a3=0.02, b3=0, c3=0 WHERE id=55584;
UPDATE uesp_esolog.minedSkills SET a1=0.04, b1=0, c1=0, type2=4, a2=0.04, b2=0, c2=0, a3=0.04, b3=0, c3=0 WHERE id=55676;
:* Fix the Consuming Trap coefficients:
UPDATE uesp_esolog.minedSkills SET type4=4,a4=0.17,b4=0,c4=0,r4=1 WHERE id=40317;
UPDATE uesp_esolog.minedSkills SET type4=4,a4=0.18,b4=0,c4=0,r4=1 WHERE id=43071;
UPDATE uesp_esolog.minedSkills SET type4=4,a4=0.19,b4=0,c4=0,r4=1 WHERE id=43077;
UPDATE uesp_esolog.minedSkills SET type4=4,a4=0.20,b4=0,c4=0,r4=1 WHERE id=43083;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.17, b=0, c=0, r=1 WHERE abilityId=40317 AND idx=5;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.18, b=0, c=0, r=1 WHERE abilityId=43071 AND idx=5;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.19, b=0, c=0, r=1 WHERE abilityId=43077 AND idx=5;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.20, b=0, c=0, r=1 WHERE abilityId=43083 AND idx=5;

== Export Game Data ==
:* Edit ''createNewExport.sh'' and update the top 4 lines with the relevant update numbers
:* Example for Live Update (using update 17):
VERSION="17" # Game update number
ISPTS="" # 1 for PTS, blank for Live
LASTVERSION="16" # The last game update
LASTPTSVERSION="17pts" # The last PTS game update
:* Example for PTS Updates (using update 17pts);
VERSION="17pts" # Game update number
ISPTS="1" # 1 for PTS, blank for Live
LASTVERSION="16" # The last game update
LASTPTSVERSION="16pts" # The last PTS game update
:* Run ''createNewExport.sh'' (typically takes 4-6 hours to run, creating 1000k files, using 200GB).
:* Fix/address any errors that occur during the script.

== Update Log Parser ==
:* Edit the ''parseLog.php'' file.
:* Update the ''START_MINEITEM_TIMESTAMP'' and real-time timestamps.
:* Update the ''MINEITEM_TABLESUFFIX'' and ''SKILLS_TABLESUFFIX'' constants with the current version.
:* (Optional) Update the ''$IGNORE_LOGENTRY_BEFORE_TIMESTAMP1'' field with the relevant timestamp. Typically this is done in conjunction with clearing the ''logEntry'' table every 6 months or so to prevent that table from becoming too large and slowing down parsing.

== Update ''esofiles.uesp.net'' ==
:* Create new directory ''content4:/home/uesp/esofiles/update-XX/''
:* Upload all ZIP files created in the export process.

== Update ''esolog/esoCommon.php'' ==
:* Add API version to ''$APIVERSION_TO_GAMEUPDATE'' and ''$APIVERSION_TO_GAMEVERSION'' arrays.
:* '''PTS'''
::* Update the ''GetEsoItemTableSuffix()'' function in ''esoCommon.php'' with the latest update numbers. Include the future live update version as follows:
case '##pts':
return "##pts";
case '##':
return "##";
:* '''Live'''
::* Once item mining and skills are complete and live update the following in ''esoCommon.php''.
::* Update the ''GetEsoItemTableSuffix()'' function to return the correct current update number.
::* Update the ''GetEsoUpdateVersion()'' function to return the current update number as follows.
case '##pts':
return "##pts";
case '##':
return "";
:* Update set indexes. Copy content from the ''Sets.php'' file generated by the data extraction script and copy into ''esoCommon.php'' replacing the existing ''$ESO_SET_INDEXES'' variable.
:* Optional: Edit and run ''updateSetIds.php'' if you need to update the internal game set ID in the setSummary table after you run ''createSetSummary.php'' or in another table.

== Skills and Champion Points ==
:* Only dump data on a character with no CPs/skills purchased and wearing no items that could affect the mined data. Also any add-ons that modify skill icons (ex: LuiExtended) should be disabled.
:* ''/uespdump skills abilities VERSION''
:* ''/uespdump skills types VERSION''
:* ''/uespdump skills crafted VERSION''
:* For Each Class: ''/uespdump skills classcrafted VERSION''
:* ''/uespdump cp VERSION''
:* Upload saved variable files.
:* Run log parser.
:* Only for Update 28 and Earlier: Edit ''createCpFits.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Edit ''fixSkillsSlotted.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Edit ''updateCraftedSkillIds.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Edit ''createSkillTree.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Edit ''fixDestructionSkills.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Edit ''fixPvpArtifactSkills.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Update and run ''php71 generateSkillIcons.php'' to create skill icons for tooltips API (after icons are uploaded).
:* For Live updates change the ''current'' symlink to the new update directory:
cd /mnt/uesp/esogameicons/uespskills
ln -sfn ./41 ./current
:* Update raw skill data and tooltips:
::* The first step can be done as soon as the files are extracted from eso.mnf.
::* Run TestSkillFormat to create the ''esoRawSkillData.php'' file from the skill data in ''eso000.mnf'' (modify the one input parameter to be ''VERSION'').
::* Copy to the ''esolog/archive/esoRawSkillDataVERSION.php'' file.
::* Update table suffix at top of ''parseRawSkillData.php''.
::* The next step has to be done after the skill coefficients have been updated and fixed (see below section for details).
::* Run ''parseRawSkillData.php''. Pay attention and fix any warnings/errors displayed (I usually only fix player skills afterwards from the database since there tends to be a lot of minor errors).

::* Run the following commands in ESO to update the flag data:
:::* Reset all CP/skills and remove equipment.
:::* Buy CPs needed for the various commands below. Remember to unequip prior CPs at each step.
:::* Run: ''/usc savedesc'' (this may need to be done at multiple points)
:::* Equip the +DOT Damage CP and run: ''/usc checkdesc dot''
:::* Equip the +DIRECT Damage CP and run: ''/usc checkdesc direct''
:::* Equip the +ST Damage CP and run: ''/usc checkdesc stdmg''
:::* Equip the +AOE Damage CP and run: ''/usc checkdesc aoedmg''
:::* Equip the +DS Duration CP (Shield Master), equip and run: ''/usc checkdesc ds''
:::* Equip the +DS CP (Bastion) and run: ''/usc checkdesc ds''
:::* Buy the +HEALING DONE CP and run: ''/usc checkdesc heal'' (keep this purchased during the next healing steps)
:::* Run: ''/usc savedesc''
:::* Equip the +HOT Healing CP and run: ''/usc checkdesc hot''
:::* Equip the +AOE Healing CP and run: ''/usc checkdesc aoeheal''
:::* Equip the +ST Healing CP and run: ''/usc checkdesc stheal''
:::* '''Not Needed Since Update 32:''' Buy the DK World in Ruin passive and run: ''/usc checkdesc flameaoe''
:::* Equip the Elf Bane set and run while in combat: ''/usc checkdesc elfbane''
:::* Equip the Diamond’s Victory set and run while in combat (trigger with a ranged spell): ''/usc checkdesc melee''
:::* Upload log and parse.

::* After Items Mined: Run: ''findSetBonusSkills.php'' to link scaling sets to their matching skill.
::* Edit version and Run: ''php71 createCraftedSkillTooltips.php''

:* (Live Only) Move old tables (minedSkills, skillTree, skillTooltips, minedSkillLines, cpSkills, cpSkillDescriptions, cpDisciplines) to ''tableOLDVERSION'' and new tables to tables with no version suffix, for example:
RENAME TABLE minedSkills TO minedSkills16; RENAME TABLE minedSkills17 TO minedSkills;
::Can now do this using the script (''VERSION'' is the current update version):
php renameTables.php skill VERSION

== Update Skill Coefficients ==
:* Only dump data on a character with no CPs/skills purchased and wearing no items.
:* When equipping items only use items/sets with "simple" modifiers. For example, a set that increases spell damage of Fire based skills will affect skills differently and doesn't update the character's weapon/spell damage stat and may result in inaccurate coefficients. Prefer items/sets that just modify base stats and apply to everything. Avoid any +healing, +damage shield, and any cost modifiers (including traits).
:* Add all skills that have a description containing a number:
/usc addall
/usc addcrafted
:* Use an add-on like Alpha-Gear to save item setups. On PTS this doesn't always work since characters are copied/deleted frequently.
:* Equip a complete setup of Light Armor and run the command:
/usc save
:: Wait for the save command to finish before running the next command.
:* Unequip 2 random pieces of armor, save, and repeat until naked. This gives around 6 points of data for each Light/Medium/Heavy setup which is a good balance between too few points (less accuracy) and too many points (more accuracy but time and space consuming).
:* Repeat with a complete setup of Medium and Heavy armor.
:* Run the command:
/usc calc
:* Reload the UI. Copy/backup the (large) saved variable file.
:* Run the command:
/usc reset
:* Reload the UI and upload the (smaller) saved variable file to the parser. Parse the new data.
:* Check for coefficients that didn't compute correctly. This regularly occurs due to skills being updated/changed. Easiest way to directly query the database using a query like:
SELECT id, name, rank FROM minedSkillsXX WHERE isPlayer=1 AND R1<0.99 AND R1>0;
::Repeat the query for R2...R6. Note that R values from 0.90-0.99 might be considered "valid" or "good" in some cases.
:* Fix the skill coefficient data found in the ''uespLog.SKILLCOEF_SPECIALTYPES'' variable within ''uespSkillCoef.lua'' as needed.
:* Restore the large saved variable backup file in order to recalculate coefficients (or simply redo all steps from scratch).
:* Recalculate the skill coefficients by running the commands:
/uespreset log
/uespreset temp
/usc calc
::Repeat the backup/upload/parse procedure until all bad coefficients have been fixed.

== Mine Item Data ==
:* Make sure you do this on a max level character (CP160) with no CP/skills purchased and not wearing any ability altering equipment.

=== PTS ===
:* Only the min/max item data are exported in PTS updates to minimize the time and storage required.
:* Initialize the item summary data (used for detecting item data corruption):
/umi createsummary
/umi checksummary
::If there are any issues detected in the last command you may need to restart the client and re-create the summary data.
:* Run the following in-game commands:
/umi table pts
/umi autostart 1
:* After each automatic /reloadui copy the uespLog.lua saved variables to a new file for later manual upload (or do it automatically with uespLogMonitor).
:* Repeat the mining for potion data.
/ud uespLog.MinePotionData()
/umi table safe
/umi potion on
/umi autostart 0
:* Item mining for PTS takes about 2 hours and there should be around 16 saved variable files.
:* Run ''/ud uespLog.DumpSetInfos(VERSION)''
:* Upload files and parse.
:* Edit ''createMinedItemSummary.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run.
:* Edit ''createSetSummary.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run.
:* Edit ''updateSetIds.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run (do this after updating ''$ESO_SET_INDEXES'').

=== Live ===
:* Initialize the item summary data (used for detecting item data corruption):
/umi createsummary
/umi checksummary
::If there are any issues detected in the last command you may need to restart the client and re-create the summary data.
:* Run the following in-game commands (make sure level/type/item type/potion options are turned off):
/umi table safe
/umi autostart 1
:* Note that the entire process takes around 1 week of full time mining to complete.
:* Repeat the mining for potion data.
/ud uespLog.MinePotionData()
/umi table safe
/umi potion on
/umi autostart 0
:* Save the valid item data by the command (replace XX with the current version):
/umi idcheck XX
:* Run ''/ud uespLog.DumpSetInfos(VERSION)''
:* Edit ''createMinedItemSummary.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run.
:* Edit ''createSetSummary.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run.
:* Edit ''updateSetIds.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run (do this after updating ''$ESO_SET_INDEXES'').
:* Delete ''fixitems.lua''.
:* Update version in ''findMissingItems.php'' and run.
:* Copy output found in ''fixitems.lua'' to ''uespLog.lua'' and run ''/ud uespLog.StartMineTest(1, false)'' in-game to mine those missing items. Upload log, parse and re-run item summary scripts.
:* Repeat for ''findBadItemSets.php'', ''findBadItemSummaries.php'', ''findBadItemSetDescs.php'', and ''findBadItemNames.php'' until no more bad items are found.
:* Run: ''findSetBonusSkills.php'' to link scaling sets to their matching skill.
:* Move old tables (minedItem, minedItemSummary, setSummary) to ''tableOLDVERSION'' and new tables to tables with no version suffix, for example:
RENAME TABLE minedItem TO minedItemXX; RENAME TABLE minedItemYY TO minedItem;
::Can now do this using the script (version is the current update version):
php renameTables.php item VERSION
:: You may need to restart Apache on content3.

== Update Log Viewer (PTS Only) ==
:* Only needed for PTS updates.
:* Edit ''viewLog.php'' and change the ''ENABLE_PTS_VERSION'' constant to the current update version.

== Update Maps ==
:* Create a 400% for all combined maps (this can be done any number of ways, the PhotoShop version is given below):
::* (Do Once) Create an action to resize 400% using "Preserve Details 2.0" resampling with 100% reduce noise.
::* Run an Image Processor using the previous action on all images in ''goodimages-VERSION\CombinedMaps'' and output to ''goodimages-VERSION\LargeMaps''.
::* Typically takes 2-3 hours to generate all maps.
::* Usually only have to do new/updated maps found in ''CombinedMapsNew'' or ''CombinedMapsNewPts''.
:* Create the zoom levels 11 and 12:
::* Run the Python script: ''CreateTilesFromLargeMap.py VERSION e:\esoexport\''
::* Copy tiles to NewMaps: ''./export/CopyNewMapTiles1112.sh VERSION''
:* Upload any files in the ''NewMaps'' folder to ''content1/2/3:/home/uesp/www/esomap/'' and ''files1:/shared/uesp/maps/esomap/''.
rsync -avz ./ dave@ssh-dev1.uesp.net:/home/uesp/www/esomap/
chown dave:uespadmin -R /home/uesp/www/esomap/
chmod g+rwx,o-w -R /home/uesp/www/esomap/
rsync -avz /home/uesp/www/esomap/ dave@content1.uesp.net:/home/uesp/www/esomap/
rsync -avz /home/uesp/www/esomap/ dave@content2.uesp.net:/home/uesp/www/esomap/
rsync -avz /home/uesp/www/esomap/ dave@content3.uesp.net:/home/uesp/www/esomap/
rsync -avz /home/uesp/www/esomap/ dave@files1.uesp.net:/shared/uesp/maps/esomap/
:* Edit ''maps_new.txt'' that contains a list of all new maps to add.
::* Edit the second column to be the displayed name.
::* Edit the last column to be the ID of the parent world. For new worlds change this to be -2 so the database can be manually updated later. Otherwise enter the worldId as found from the ESO map if applicable.
:* On db1 find the last user world ID by the SQL command: ''select max(id) from uesp_gamemap.world'';
:* Edit ''createNewMaps.sh'' with the first free world ID (+1 from the previous step) and update version and run.
:* Upload the ''newmaps.sql'' to db1.
:* Backup the current ''uesp_gamemap'' database:
mysqldump --opt -u user -p uesp_gamemap > gamemap.sql
:* Run the SQL file:
mysql -u user -p uesp_gamemap < newmaps.sql
:* Run ''php makeZoomLinks.php'' on files1 to update Leaflet symlinks for the new map.
:* Test the map to ensure new maps have been successfully added.

== Update Game Data ==

=== Achievements ===
:* This is usually only done for Live updates.
:* ''/uespdump achievements VERSION''
:* Upload log and run log parser.
:* Copy old ''esoAchievementData.php'' to ''esoAchievementDataXX.php''
:* Run ''createAchievementData.php''

=== Collectibles ===
:* This is usually only done for Live updates but can be done for PTS updates in order to fully update the uespLog runebox data.
:* ''/uespminecollect''
:* ''/ud uespLog.MineCollectibleTree()''
:* Run createRuneBoxData.php (after mined items are updated) and update ''uespLog.RUNEBOX_COLLECTIBLE_IDS'' array in ''uespLog.lua'' with any new runebox/collectible pairs.
:* Upload log and run log parser.
:* (Live Only) Copy old esoCollectibleData.php to esoCollectibleDataXX.php. Copy data out of ''tempData'' in the ''uespLog.lua'' saved variable file and re-format/paste into ''esoCollectibleData.php''.

=== Books ===
:* This is usually only done for Live updates.
:* Perform this on a character that has unlocked Eidetic Memory/Shalidor's Library.
:* Run ''/ud uespLog.MineBookTree()'' in-game.
:* Run ''/ud uespLog.MineBooks()'' in-game (upload/parse log data).
:* Copy data from the ''tempData'' section of the saved variable file, re-format, and paste into ''esoBookCollectionData.php''.
:* Upload ''books.sql'' to db1 and run.

=== Quests ===
:* This is usually only done for Live updates.
:* Update the ''esoQuestData.php'' file from the ''Quests/Quest.php'' file created by the export script.
:* NOTE: This step is no longer required due to a change in quest parsing.
::* Upload ''quests.sql'' to db1 and run.

=== Recipes ===
:* This is usually only done for Live updates.
:* Perform this on a PTS template to ensure all recipes are recorded correctly.
:* Run ''/ud uespLog.MineRecipeDataStart()'' from in-game.
:* Copy data from the ''tempData'' section of the saved variable file, re-format, and paste into ''esoRecipeData.php''.

=== Sets ===
:* Update the ''setInfo'' table column ''sources'' (other data should be updated from the game itself).
:* Edit and run ''generateSetImages.php'' to create image tooltips if needed.

=== Styles / Motifs ===
:* This only has to be done if new motifs/styles were added or existing styles changed.
:* Use ''/ud GetHighestItemStyleId()'' in-game to get the last style ID.
:* Use ''/ud GetItemStyleName(#)'' in-game to get the style material.
:* Use ''/ud GetItemStyleMaterialLink(#)'' in-game to get the style material and find the icon name.
:* Use ''/uespstyle all'', ''/uespstyle allid'' in-game to list all styles.
:* Edit ''uespLog.lua'' and update the ''uespLog.CRAFTSTYLENAME_TO_ITEMSTYLE'' and ''uespLog.CRAFTSTYLENAME_TO_MOTIFID'' arrays.
:* Edit ''uespLogTradeData.lua'' and update the ''uespLog.ALT_STYLE_ICON_DATA'' and ''uespLog.STYLE_ICON_DATA'' arrays.
:* Edit ''esolog:esoCommon.php'' and update the ''$ESO_ITEMSTYLE_TEXTS'' array.
:* Edit ''esochardata:viewCharData.class.php'' and update the ''$MOTIFS_FOR_MASTERWRITS'' array.

=== Traits ===
:* This only has to be done when new traits are added.
:* Edit ''esolog:esoCommon.php'' and update the ''$ESO_ITEMTRAIT##_FULLTEXTS'', ''$ESO_ITEMTRAIT_DESCRIPTIONS'', ''$ESO_ITEMTRANSMUTETRAIT_IDS'', and ''$ESO_ITEMTRAIT##_TEXTS'' arrays.
:* Edit ''esolog:resources/esoitemlink.js'' and update the ''ESO_ITEM_TRAITS'' array.
:* Edit ''esolog:resources/esoItemSearchPopup.js'' and update the ''UESP.ESO_TRAIT_TYPES'' array (and select lists if needed).

=== Antiquities ===
:* Usually only done for live updates.
:* Run ''/ud uespLog.MineAntiquities()'' in-game and upload/parse the log file.

=== Other Data ===
:* ''esolog:esoCommon.php $ESO_ITEMTYPE_TEXTS''
:* ''esolog:esoCommon.php $ESO_ITEMSPECIALTYPE_TEXTS'' and ''$ESO_ITEMSPECIALTYPE_RAW_TEXTS''
::* Some constants (styles, item type, item special type, item display category, and quest type) can be checked by running the ''/ud uespLog.MineApiConstants()'' command and then extracting the text from the ''tempData'' section of the saved variables file.
:* ''esolog:esoCommon.php $ESO_SET_INDEXES'' (from ''Sets.php'' created by the ''createNewExport.sh'' script)

== Update Icons ==
:* Upload art to content3:
rsync -avz ./esomnf-XX/esoui/art/ dave@ssh-content3.uesp.net:/home/uesp/www/eso/gameicons/esoui/art/
rsync -avz ./gamemnf-XX/esoui/art/ dave@ssh-content3.uesp.net:/home/uesp/www/eso/gameicons/esoui/art/
:* Update file permissions:
sudo chown dave:uespadmin -R /home/uesp/www/eso/gameicons/
sudo chmod a+rx,o-w -R /home/uesp/www/eso/gameicons/
:* Copy from content3 to files1:
rsync -avz --checksum /home/uesp/www/eso/gameicons/ dave@files1.uesp.net:/shared/uesp/esogameicons/

== Update API ==
:* This is only done for Live updates.
:* Run: ''/uespdump globals 7'' in-game.
:* Copy just the ''['globals']'' section out from the saved variable file into a ''globals.txt'' file.
:* Remove the leading spaces/tabs but keep the Lua array format. All lines should look like:
[2] = "event{Global} ... lang{en} ",
:* Ensure your file system is setup with case-insensitive filenames. (see below for Windows).
:* Modify ''EsoParseData.py'' and update the ''INPUT_GLOBAL_FILENAME'', ''INPUT_LUA_PATH'', and ''OUTPUT_PATH'' variables.
:* Run ''EsoParseData.py'' from a CygWin shell window.
:* If script aborts in error fix the folder security permissions in Windows and retry.
:* Edit ''makeApiDiff.sh'' to the current and previous API versions and run.
:* Upload the new API directory to ''content2:/home/uesp/www/eso/data/''.
rsync -avz ./101047 dave@ssh-content2.uesp.net:/home/uesp/www/eso/data/
:* Copy ''api.html'' to the root ''index.html''.
:* Replace the symbolic link ''current'' to the new API directory.
ln -sfn 100025 current

== Update Sales Items ==
:* Only done for Live updates.
:* Once items have been mined and are live run the script:
php fixNewSalesItems.php
:: to update the data of any new items that have appeared in sales data since the update.

== Other Things ==
:* Clear the CloudFlare cache once all update items have been completed. Otherwise map, icons, and other resources may still show the prior version for some time.

== TODO ==

== Notes ==

:* '''Modify Windows to use case insensitive filenames in Cygwin shell.'''
::* Install the Linux Subsystem in Windows.
::* Run the command:
fsutil.exe file setCaseSensitiveInfo "full path of folder" enable
::* This doesn't change any existing sub-folders but should enable it on any new sub-folders created after the command is run.
::* Old Method (No Longer Works) As of ~2018 the below method seems to no longer work.
:::* Set Windows registry key ''HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\obcaseinsensitive=dword:00000000''.
:::* Reboot after modifying registry key.
:::* Use Cygwin with ''posix=1'' set for all relevant drives in ''/etc/fstab''.
:::* Test case insensitivity before running script.
:::* (Optional) Set Windows registry key ''obcaseinsensitive'' back 1 after finished.

User:Daveh/ESO Update

2026-05-01T13:41:46Z

Daveh: /* Update API */

This page details the procedure of updating the UESP data after a major ESO update/release.

== General Prep and Notes ==
:* Run ESO patcher to get latest version.
:* (Optional) Edit ''content3:/home/uesp/esolog/esolog.index'' and increment/note the log index. Optional but useful if you need to find a specific entry in the new logs.
:* (Optional) Backup/delete the ''uespLog_backupData.txt'' file created by uespLogMonitor.
:* PTS/Live updates are slightly different. Most steps are for both updates while some steps are for only one or the other.
:* Note that the ''VERSION'' field used for scripts/databases is just the update number itself or the update number with pts appended to it. For example:
::* Live: 17
::* PTS: 17pts

== Special Notes ==
Things that need manually fixing for each update:
:* Fix the Undaunted Command stamina coefficients:
UPDATE uesp_esolog.skillTooltips SET coefType=32, a=0.02, b=0, c=0, r=1 WHERE abilityId=55584 AND idx=1;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.02, b=0, c=0, r=1 WHERE abilityId=55584 AND idx=2;
UPDATE uesp_esolog.skillTooltips SET coefType=1, a=0.02, b=0, c=0, r=1 WHERE abilityId=55584 AND idx=3;
UPDATE uesp_esolog.skillTooltips SET coefType=32, a=0.04, b=0, c=0, r=1 WHERE abilityId=55676 AND idx=1;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.04, b=0, c=0, r=1 WHERE abilityId=55676 AND idx=2;
UPDATE uesp_esolog.skillTooltips SET coefType=1, a=0.04, b=0, c=0, r=1 WHERE abilityId=55676 AND idx=3;
UPDATE uesp_esolog.minedSkills SET a1=0.02, b1=0, c1=0, type2=4, a2=0.02, b2=0, c2=0, a3=0.02, b3=0, c3=0 WHERE id=55584;
UPDATE uesp_esolog.minedSkills SET a1=0.04, b1=0, c1=0, type2=4, a2=0.04, b2=0, c2=0, a3=0.04, b3=0, c3=0 WHERE id=55676;
:* Fix the Consuming Trap coefficients:
UPDATE uesp_esolog.minedSkills SET type4=4,a4=0.17,b4=0,c4=0,r4=1 WHERE id=40317;
UPDATE uesp_esolog.minedSkills SET type4=4,a4=0.18,b4=0,c4=0,r4=1 WHERE id=43071;
UPDATE uesp_esolog.minedSkills SET type4=4,a4=0.19,b4=0,c4=0,r4=1 WHERE id=43077;
UPDATE uesp_esolog.minedSkills SET type4=4,a4=0.20,b4=0,c4=0,r4=1 WHERE id=43083;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.17, b=0, c=0, r=1 WHERE abilityId=40317 AND idx=5;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.18, b=0, c=0, r=1 WHERE abilityId=43071 AND idx=5;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.19, b=0, c=0, r=1 WHERE abilityId=43077 AND idx=5;
UPDATE uesp_esolog.skillTooltips SET coefType=4, a=0.20, b=0, c=0, r=1 WHERE abilityId=43083 AND idx=5;

== Export Game Data ==
:* Edit ''createNewExport.sh'' and update the top 4 lines with the relevant update numbers
:* Example for Live Update (using update 17):
VERSION="17" # Game update number
ISPTS="" # 1 for PTS, blank for Live
LASTVERSION="16" # The last game update
LASTPTSVERSION="17pts" # The last PTS game update
:* Example for PTS Updates (using update 17pts);
VERSION="17pts" # Game update number
ISPTS="1" # 1 for PTS, blank for Live
LASTVERSION="16" # The last game update
LASTPTSVERSION="16pts" # The last PTS game update
:* Run ''createNewExport.sh'' (typically takes 4-6 hours to run, creating 1000k files, using 200GB).
:* Fix/address any errors that occur during the script.

== Update Log Parser ==
:* Edit the ''parseLog.php'' file.
:* Update the ''START_MINEITEM_TIMESTAMP'' and real-time timestamps.
:* Update the ''MINEITEM_TABLESUFFIX'' and ''SKILLS_TABLESUFFIX'' constants with the current version.
:* (Optional) Update the ''$IGNORE_LOGENTRY_BEFORE_TIMESTAMP1'' field with the relevant timestamp. Typically this is done in conjunction with clearing the ''logEntry'' table every 6 months or so to prevent that table from becoming too large and slowing down parsing.

== Update ''esofiles.uesp.net'' ==
:* Create new directory ''content4:/home/uesp/esofiles/update-XX/''
:* Upload all ZIP files created in the export process.

== Update ''esolog/esoCommon.php'' ==
:* Add API version to ''$APIVERSION_TO_GAMEUPDATE'' and ''$APIVERSION_TO_GAMEVERSION'' arrays.
:* '''PTS'''
::* Update the ''GetEsoItemTableSuffix()'' function in ''esoCommon.php'' with the latest update numbers. Include the future live update version as follows:
case '##pts':
return "##pts";
case '##':
return "##";
:* '''Live'''
::* Once item mining and skills are complete and live update the following in ''esoCommon.php''.
::* Update the ''GetEsoItemTableSuffix()'' function to return the correct current update number.
::* Update the ''GetEsoUpdateVersion()'' function to return the current update number as follows.
case '##pts':
return "##pts";
case '##':
return "";
:* Update set indexes. Copy content from the ''Sets.php'' file generated by the data extraction script and copy into ''esoCommon.php'' replacing the existing ''$ESO_SET_INDEXES'' variable.
:* Optional: Edit and run ''updateSetIds.php'' if you need to update the internal game set ID in the setSummary table after you run ''createSetSummary.php'' or in another table.

== Skills and Champion Points ==
:* Only dump data on a character with no CPs/skills purchased and wearing no items that could affect the mined data. Also any add-ons that modify skill icons (ex: LuiExtended) should be disabled.
:* ''/uespdump skills abilities VERSION''
:* ''/uespdump skills types VERSION''
:* ''/uespdump skills crafted VERSION''
:* For Each Class: ''/uespdump skills classcrafted VERSION''
:* ''/uespdump cp VERSION''
:* Upload saved variable files.
:* Run log parser.
:* Only for Update 28 and Earlier: Edit ''createCpFits.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Edit ''fixSkillsSlotted.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Edit ''updateCraftedSkillIds.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Edit ''createSkillTree.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Edit ''fixDestructionSkills.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Edit ''fixPvpArtifactSkills.php'' with the new table ''$TABLE_SUFFIX'' and run.
:* Update and run ''php71 generateSkillIcons.php'' to create skill icons for tooltips API (after icons are uploaded).
:* For Live updates change the ''current'' symlink to the new update directory:
cd /mnt/uesp/esogameicons/uespskills
ln -sfn ./41 ./current
:* Update raw skill data and tooltips:
::* The first step can be done as soon as the files are extracted from eso.mnf.
::* Run TestSkillFormat to create the ''esoRawSkillData.php'' file from the skill data in ''eso000.mnf'' (modify the one input parameter to be ''VERSION'').
::* Copy to the ''esolog/archive/esoRawSkillDataVERSION.php'' file.
::* Update table suffix at top of ''parseRawSkillData.php''.
::* The next step has to be done after the skill coefficients have been updated and fixed (see below section for details).
::* Run ''parseRawSkillData.php''. Pay attention and fix any warnings/errors displayed (I usually only fix player skills afterwards from the database since there tends to be a lot of minor errors).

::* Run the following commands in ESO to update the flag data:
:::* Reset all CP/skills and remove equipment.
:::* Buy CPs needed for the various commands below. Remember to unequip prior CPs at each step.
:::* Run: ''/usc savedesc'' (this may need to be done at multiple points)
:::* Equip the +DOT Damage CP and run: ''/usc checkdesc dot''
:::* Equip the +DIRECT Damage CP and run: ''/usc checkdesc direct''
:::* Equip the +ST Damage CP and run: ''/usc checkdesc stdmg''
:::* Equip the +AOE Damage CP and run: ''/usc checkdesc aoedmg''
:::* Equip the +DS Duration CP (Shield Master), equip and run: ''/usc checkdesc ds''
:::* Equip the +DS CP (Bastion) and run: ''/usc checkdesc ds''
:::* Buy the +HEALING DONE CP and run: ''/usc checkdesc heal'' (keep this purchased during the next healing steps)
:::* Run: ''/usc savedesc''
:::* Equip the +HOT Healing CP and run: ''/usc checkdesc hot''
:::* Equip the +AOE Healing CP and run: ''/usc checkdesc aoeheal''
:::* Equip the +ST Healing CP and run: ''/usc checkdesc stheal''
:::* '''Not Needed Since Update 32:''' Buy the DK World in Ruin passive and run: ''/usc checkdesc flameaoe''
:::* Equip the Elf Bane set and run while in combat: ''/usc checkdesc elfbane''
:::* Equip the Diamond’s Victory set and run while in combat (trigger with a ranged spell): ''/usc checkdesc melee''
:::* Upload log and parse.

::* After Items Mined: Run: ''findSetBonusSkills.php'' to link scaling sets to their matching skill.
::* Edit version and Run: ''php71 createCraftedSkillTooltips.php''

:* (Live Only) Move old tables (minedSkills, skillTree, skillTooltips, minedSkillLines, cpSkills, cpSkillDescriptions, cpDisciplines) to ''tableOLDVERSION'' and new tables to tables with no version suffix, for example:
RENAME TABLE minedSkills TO minedSkills16; RENAME TABLE minedSkills17 TO minedSkills;
::Can now do this using the script (''VERSION'' is the current update version):
php renameTables.php skill VERSION

== Update Skill Coefficients ==
:* Only dump data on a character with no CPs/skills purchased and wearing no items.
:* When equipping items only use items/sets with "simple" modifiers. For example, a set that increases spell damage of Fire based skills will affect skills differently and doesn't update the character's weapon/spell damage stat and may result in inaccurate coefficients. Prefer items/sets that just modify base stats and apply to everything. Avoid any +healing, +damage shield, and any cost modifiers (including traits).
:* Add all skills that have a description containing a number:
/usc addall
/usc addcrafted
:* Use an add-on like Alpha-Gear to save item setups. On PTS this doesn't always work since characters are copied/deleted frequently.
:* Equip a complete setup of Light Armor and run the command:
/usc save
:: Wait for the save command to finish before running the next command.
:* Unequip 2 random pieces of armor, save, and repeat until naked. This gives around 6 points of data for each Light/Medium/Heavy setup which is a good balance between too few points (less accuracy) and too many points (more accuracy but time and space consuming).
:* Repeat with a complete setup of Medium and Heavy armor.
:* Run the command:
/usc calc
:* Reload the UI. Copy/backup the (large) saved variable file.
:* Run the command:
/usc reset
:* Reload the UI and upload the (smaller) saved variable file to the parser. Parse the new data.
:* Check for coefficients that didn't compute correctly. This regularly occurs due to skills being updated/changed. Easiest way to directly query the database using a query like:
SELECT id, name, rank FROM minedSkillsXX WHERE isPlayer=1 AND R1<0.99 AND R1>0;
::Repeat the query for R2...R6. Note that R values from 0.90-0.99 might be considered "valid" or "good" in some cases.
:* Fix the skill coefficient data found in the ''uespLog.SKILLCOEF_SPECIALTYPES'' variable within ''uespSkillCoef.lua'' as needed.
:* Restore the large saved variable backup file in order to recalculate coefficients (or simply redo all steps from scratch).
:* Recalculate the skill coefficients by running the commands:
/uespreset log
/uespreset temp
/usc calc
::Repeat the backup/upload/parse procedure until all bad coefficients have been fixed.

== Mine Item Data ==
:* Make sure you do this on a max level character (CP160) with no CP/skills purchased and not wearing any ability altering equipment.

=== PTS ===
:* Only the min/max item data are exported in PTS updates to minimize the time and storage required.
:* Initialize the item summary data (used for detecting item data corruption):
/umi createsummary
/umi checksummary
::If there are any issues detected in the last command you may need to restart the client and re-create the summary data.
:* Run the following in-game commands:
/umi table pts
/umi autostart 1
:* After each automatic /reloadui copy the uespLog.lua saved variables to a new file for later manual upload (or do it automatically with uespLogMonitor).
:* Repeat the mining for potion data.
/ud uespLog.MinePotionData()
/umi table safe
/umi potion on
/umi autostart 0
:* Item mining for PTS takes about 2 hours and there should be around 16 saved variable files.
:* Run ''/ud uespLog.DumpSetInfos(VERSION)''
:* Upload files and parse.
:* Edit ''createMinedItemSummary.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run.
:* Edit ''createSetSummary.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run.
:* Edit ''updateSetIds.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run (do this after updating ''$ESO_SET_INDEXES'').

=== Live ===
:* Initialize the item summary data (used for detecting item data corruption):
/umi createsummary
/umi checksummary
::If there are any issues detected in the last command you may need to restart the client and re-create the summary data.
:* Run the following in-game commands (make sure level/type/item type/potion options are turned off):
/umi table safe
/umi autostart 1
:* Note that the entire process takes around 1 week of full time mining to complete.
:* Repeat the mining for potion data.
/ud uespLog.MinePotionData()
/umi table safe
/umi potion on
/umi autostart 0
:* Save the valid item data by the command (replace XX with the current version):
/umi idcheck XX
:* Run ''/ud uespLog.DumpSetInfos(VERSION)''
:* Edit ''createMinedItemSummary.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run.
:* Edit ''createSetSummary.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run.
:* Edit ''updateSetIds.php'', update the ''$TABLE_SUFFIX'' variable with the current version and run (do this after updating ''$ESO_SET_INDEXES'').
:* Delete ''fixitems.lua''.
:* Update version in ''findMissingItems.php'' and run.
:* Copy output found in ''fixitems.lua'' to ''uespLog.lua'' and run ''/ud uespLog.StartMineTest(1, false)'' in-game to mine those missing items. Upload log, parse and re-run item summary scripts.
:* Repeat for ''findBadItemSets.php'', ''findBadItemSummaries.php'', ''findBadItemSetDescs.php'', and ''findBadItemNames.php'' until no more bad items are found.
:* Run: ''findSetBonusSkills.php'' to link scaling sets to their matching skill.
:* Move old tables (minedItem, minedItemSummary, setSummary) to ''tableOLDVERSION'' and new tables to tables with no version suffix, for example:
RENAME TABLE minedItem TO minedItemXX; RENAME TABLE minedItemYY TO minedItem;
::Can now do this using the script (version is the current update version):
php renameTables.php item VERSION
:: You may need to restart Apache on content3.

== Update Log Viewer (PTS Only) ==
:* Only needed for PTS updates.
:* Edit ''viewLog.php'' and change the ''ENABLE_PTS_VERSION'' constant to the current update version.

== Update Maps ==
:* Create a 400% for all combined maps (this can be done any number of ways, the PhotoShop version is given below):
::* (Do Once) Create an action to resize 400% using "Preserve Details 2.0" resampling with 100% reduce noise.
::* Run an Image Processor using the previous action on all images in ''goodimages-VERSION\CombinedMaps'' and output to ''goodimages-VERSION\LargeMaps''.
::* Typically takes 2-3 hours to generate all maps.
::* Usually only have to do new/updated maps found in ''CombinedMapsNew'' or ''CombinedMapsNewPts''.
:* Create the zoom levels 11 and 12:
::* Run the Python script: ''CreateTilesFromLargeMap.py VERSION e:\esoexport\''
::* Copy tiles to NewMaps: ''./export/CopyNewMapTiles1112.sh VERSION''
:* Upload any files in the ''NewMaps'' folder to ''content1/2/3:/home/uesp/www/esomap/'' and ''files1:/shared/uesp/maps/esomap/''.
rsync -avz ./ dave@ssh-content3.uesp.net:/home/uesp/www/esomap/
chown dave:uespadmin -R /home/uesp/www/esomap/
chmod g+rwx,o-w -R /home/uesp/www/esomap/
rsync -avz /home/uesp/www/esomap/ dave@content1.uesp.net:/home/uesp/www/esomap/
rsync -avz /home/uesp/www/esomap/ dave@content2.uesp.net:/home/uesp/www/esomap/
rsync -avz /home/uesp/www/esomap/ dave@content4.uesp.net:/home/uesp/www/esomap/
rsync -avz /home/uesp/www/esomap/ dave@files1.uesp.net:/shared/uesp/maps/esomap/
:* Edit ''maps_new.txt'' that contains a list of all new maps to add.
::* Edit the second column to be the displayed name.
::* Edit the last column to be the ID of the parent world. For new worlds change this to be -2 so the database can be manually updated later. Otherwise enter the worldId as found from the ESO map if applicable.
:* On db1 find the last user world ID by the SQL command: ''select max(id) from uesp_gamemap.world'';
:* Edit ''createNewMaps.sh'' with the first free world ID (+1 from the previous step) and update version and run.
:* Upload the ''newmaps.sql'' to db1.
:* Backup the current ''uesp_gamemap'' database:
mysqldump --opt -u user -p uesp_gamemap > gamemap.sql
:* Run the SQL file:
mysql -u user -p uesp_gamemap < newmaps.sql
:* Run ''php makeZoomLinks.php'' on files1 to update Leaflet symlinks for the new map.
:* Test the map to ensure new maps have been successfully added.

== Update Game Data ==

=== Achievements ===
:* This is usually only done for Live updates.
:* ''/uespdump achievements VERSION''
:* Upload log and run log parser.
:* Copy old ''esoAchievementData.php'' to ''esoAchievementDataXX.php''
:* Run ''createAchievementData.php''

=== Collectibles ===
:* This is usually only done for Live updates but can be done for PTS updates in order to fully update the uespLog runebox data.
:* ''/uespminecollect''
:* ''/ud uespLog.MineCollectibleTree()''
:* Run createRuneBoxData.php (after mined items are updated) and update ''uespLog.RUNEBOX_COLLECTIBLE_IDS'' array in ''uespLog.lua'' with any new runebox/collectible pairs.
:* Upload log and run log parser.
:* (Live Only) Copy old esoCollectibleData.php to esoCollectibleDataXX.php. Copy data out of ''tempData'' in the ''uespLog.lua'' saved variable file and re-format/paste into ''esoCollectibleData.php''.

=== Books ===
:* This is usually only done for Live updates.
:* Perform this on a character that has unlocked Eidetic Memory/Shalidor's Library.
:* Run ''/ud uespLog.MineBookTree()'' in-game.
:* Run ''/ud uespLog.MineBooks()'' in-game (upload/parse log data).
:* Copy data from the ''tempData'' section of the saved variable file, re-format, and paste into ''esoBookCollectionData.php''.
:* Upload ''books.sql'' to db1 and run.

=== Quests ===
:* This is usually only done for Live updates.
:* Update the ''esoQuestData.php'' file from the ''Quests/Quest.php'' file created by the export script.
:* NOTE: This step is no longer required due to a change in quest parsing.
::* Upload ''quests.sql'' to db1 and run.

=== Recipes ===
:* This is usually only done for Live updates.
:* Perform this on a PTS template to ensure all recipes are recorded correctly.
:* Run ''/ud uespLog.MineRecipeDataStart()'' from in-game.
:* Copy data from the ''tempData'' section of the saved variable file, re-format, and paste into ''esoRecipeData.php''.

=== Sets ===
:* Update the ''setInfo'' table column ''sources'' (other data should be updated from the game itself).
:* Edit and run ''generateSetImages.php'' to create image tooltips if needed.

=== Styles / Motifs ===
:* This only has to be done if new motifs/styles were added or existing styles changed.
:* Use ''/ud GetHighestItemStyleId()'' in-game to get the last style ID.
:* Use ''/ud GetItemStyleName(#)'' in-game to get the style material.
:* Use ''/ud GetItemStyleMaterialLink(#)'' in-game to get the style material and find the icon name.
:* Use ''/uespstyle all'', ''/uespstyle allid'' in-game to list all styles.
:* Edit ''uespLog.lua'' and update the ''uespLog.CRAFTSTYLENAME_TO_ITEMSTYLE'' and ''uespLog.CRAFTSTYLENAME_TO_MOTIFID'' arrays.
:* Edit ''uespLogTradeData.lua'' and update the ''uespLog.ALT_STYLE_ICON_DATA'' and ''uespLog.STYLE_ICON_DATA'' arrays.
:* Edit ''esolog:esoCommon.php'' and update the ''$ESO_ITEMSTYLE_TEXTS'' array.
:* Edit ''esochardata:viewCharData.class.php'' and update the ''$MOTIFS_FOR_MASTERWRITS'' array.

=== Traits ===
:* This only has to be done when new traits are added.
:* Edit ''esolog:esoCommon.php'' and update the ''$ESO_ITEMTRAIT##_FULLTEXTS'', ''$ESO_ITEMTRAIT_DESCRIPTIONS'', ''$ESO_ITEMTRANSMUTETRAIT_IDS'', and ''$ESO_ITEMTRAIT##_TEXTS'' arrays.
:* Edit ''esolog:resources/esoitemlink.js'' and update the ''ESO_ITEM_TRAITS'' array.
:* Edit ''esolog:resources/esoItemSearchPopup.js'' and update the ''UESP.ESO_TRAIT_TYPES'' array (and select lists if needed).

=== Antiquities ===
:* Usually only done for live updates.
:* Run ''/ud uespLog.MineAntiquities()'' in-game and upload/parse the log file.

=== Other Data ===
:* ''esolog:esoCommon.php $ESO_ITEMTYPE_TEXTS''
:* ''esolog:esoCommon.php $ESO_ITEMSPECIALTYPE_TEXTS'' and ''$ESO_ITEMSPECIALTYPE_RAW_TEXTS''
::* Some constants (styles, item type, item special type, item display category, and quest type) can be checked by running the ''/ud uespLog.MineApiConstants()'' command and then extracting the text from the ''tempData'' section of the saved variables file.
:* ''esolog:esoCommon.php $ESO_SET_INDEXES'' (from ''Sets.php'' created by the ''createNewExport.sh'' script)

== Update Icons ==
:* Upload art to content3:
rsync -avz ./esomnf-XX/esoui/art/ dave@ssh-content3.uesp.net:/home/uesp/www/eso/gameicons/esoui/art/
rsync -avz ./gamemnf-XX/esoui/art/ dave@ssh-content3.uesp.net:/home/uesp/www/eso/gameicons/esoui/art/
:* Update file permissions:
sudo chown dave:uespadmin -R /home/uesp/www/eso/gameicons/
sudo chmod a+rx,o-w -R /home/uesp/www/eso/gameicons/
:* Copy from content3 to files1:
rsync -avz --checksum /home/uesp/www/eso/gameicons/ dave@files1.uesp.net:/shared/uesp/esogameicons/

== Update API ==
:* This is only done for Live updates.
:* Run: ''/uespdump globals 7'' in-game.
:* Copy just the ''['globals']'' section out from the saved variable file into a ''globals.txt'' file.
:* Remove the leading spaces/tabs but keep the Lua array format. All lines should look like:
[2] = "event{Global} ... lang{en} ",
:* Ensure your file system is setup with case-insensitive filenames. (see below for Windows).
:* Modify ''EsoParseData.py'' and update the ''INPUT_GLOBAL_FILENAME'', ''INPUT_LUA_PATH'', and ''OUTPUT_PATH'' variables.
:* Run ''EsoParseData.py'' from a CygWin shell window.
:* If script aborts in error fix the folder security permissions in Windows and retry.
:* Edit ''makeApiDiff.sh'' to the current and previous API versions and run.
:* Upload the new API directory to ''content2:/home/uesp/www/eso/data/''.
rsync -avz ./101047 dave@ssh-content2.uesp.net:/home/uesp/www/eso/data/
:* Copy ''api.html'' to the root ''index.html''.
:* Replace the symbolic link ''current'' to the new API directory.
ln -sfn 100025 current

== Update Sales Items ==
:* Only done for Live updates.
:* Once items have been mined and are live run the script:
php fixNewSalesItems.php
:: to update the data of any new items that have appeared in sales data since the update.

== Other Things ==
:* Clear the CloudFlare cache once all update items have been completed. Otherwise map, icons, and other resources may still show the prior version for some time.

== TODO ==

== Notes ==

:* '''Modify Windows to use case insensitive filenames in Cygwin shell.'''
::* Install the Linux Subsystem in Windows.
::* Run the command:
fsutil.exe file setCaseSensitiveInfo "full path of folder" enable
::* This doesn't change any existing sub-folders but should enable it on any new sub-folders created after the command is run.
::* Old Method (No Longer Works) As of ~2018 the below method seems to no longer work.
:::* Set Windows registry key ''HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\obcaseinsensitive=dword:00000000''.
:::* Reboot after modifying registry key.
:::* Use Cygwin with ''posix=1'' set for all relevant drives in ''/etc/fstab''.
:::* Test case insensitivity before running script.
:::* (Optional) Set Windows registry key ''obcaseinsensitive'' back 1 after finished.

UESPWiki:Administrator Noticeboard

2026-04-30T13:40:57Z

Daveh: /* Server Upgrade */ upgrade complete

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers. Sites/services will be changed and tested one-by-one and listed below once verified. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 12:55, 21 April 2026 (UTC)
::* Starfield Wiki
::* UESP Wiki
::* Gamemap
::* Fallout Wiki
::* ESO Log and related services
::* ESO API/Data
::* ESO Files
::* UESP Forums
::* CSList
::* All Construction Set related Wikis
::* UESP Blog
::* Fallout DE and Beyond Skyrim Wikis

'''Upgrade Complete''' -- All services have been moved to the new servers and everything should be working. If there are any issues in the coming weeks please report them below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:40, 30 April 2026 (UTC)

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-30T13:33:58Z

Daveh: /* Backup1 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).
::* Setup and tested cron jobs and logrotations.

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Setup and tested cron jobs and logrotations.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.
::* Setup and tested cron jobs and logrotations.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* Install MariaDB database and copy dev dbs over.
::* Setup and tested cron jobs and logrotations.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.
::* Setup and tested cron jobs and logrotations.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.
::* Setup and tested cron jobs and logrotations.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.
::* Setup and tested cron jobs and logrotations.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok
:* Setup and tested cron jobs and logrotations.

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.
::* Setup and tested cron jobs and logrotations.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.
::* Tested all cron jobs and added log rotations.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-28T15:05:38Z

Daveh: /* Search1 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).
::* Setup and tested cron jobs and logrotations.

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Setup and tested cron jobs and logrotations.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.
::* Setup and tested cron jobs and logrotations.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* Install MariaDB database and copy dev dbs over.
::* Setup and tested cron jobs and logrotations.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.
::* Setup and tested cron jobs and logrotations.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.
::* Setup and tested cron jobs and logrotations.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.
::* Setup and tested cron jobs and logrotations.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok
:* Setup and tested cron jobs and logrotations.

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.
::* Setup and tested cron jobs and logrotations.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-28T15:05:31Z

Daveh: /* Files1 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).
::* Setup and tested cron jobs and logrotations.

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Setup and tested cron jobs and logrotations.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.
::* Setup and tested cron jobs and logrotations.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* Install MariaDB database and copy dev dbs over.
::* Setup and tested cron jobs and logrotations.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.
::* Setup and tested cron jobs and logrotations.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.
::* Setup and tested cron jobs and logrotations.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.
::* Setup and tested cron jobs and logrotations.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok
:* Setup and tested cron jobs and logrotations.

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-28T14:48:53Z

Daveh: /* Db2 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).
::* Setup and tested cron jobs and logrotations.

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Setup and tested cron jobs and logrotations.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.
::* Setup and tested cron jobs and logrotations.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* Install MariaDB database and copy dev dbs over.
::* Setup and tested cron jobs and logrotations.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.
::* Setup and tested cron jobs and logrotations.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.
::* Setup and tested cron jobs and logrotations.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.
::* Setup and tested cron jobs and logrotations.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-28T14:48:48Z

Daveh: /* Db1 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).
::* Setup and tested cron jobs and logrotations.

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Setup and tested cron jobs and logrotations.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.
::* Setup and tested cron jobs and logrotations.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* Install MariaDB database and copy dev dbs over.
::* Setup and tested cron jobs and logrotations.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.
::* Setup and tested cron jobs and logrotations.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.
::* Setup and tested cron jobs and logrotations.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-28T14:42:35Z

Daveh: /* Content1 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).
::* Setup and tested cron jobs and logrotations.

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Setup and tested cron jobs and logrotations.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.
::* Setup and tested cron jobs and logrotations.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* Install MariaDB database and copy dev dbs over.
::* Setup and tested cron jobs and logrotations.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.
::* Setup and tested cron jobs and logrotations.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-28T14:42:27Z

Daveh: /* Content3 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Setup and tested cron jobs and logrotations.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.
::* Setup and tested cron jobs and logrotations.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* Install MariaDB database and copy dev dbs over.
::* Setup and tested cron jobs and logrotations.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.
::* Setup and tested cron jobs and logrotations.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-28T14:42:22Z

Daveh: /* Content2 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Setup and tested cron jobs and logrotations.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* Install MariaDB database and copy dev dbs over.
::* Setup and tested cron jobs and logrotations.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.
::* Setup and tested cron jobs and logrotations.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-27T16:22:15Z

Daveh: /* Cache1 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* Install MariaDB database and copy dev dbs over.
::* Setup and tested cron jobs and logrotations.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.
::* Setup and tested cron jobs and logrotations.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-27T16:21:44Z

Daveh: /* Dev1 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* Install MariaDB database and copy dev dbs over.
::* Setup and tested cron jobs and logrotations.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-26T13:48:14Z

Daveh: /* Post Upgrade Issues */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.
:* PHP 7.4 appears to use a lot more memory than the old servers. https://esolog.uesp.net/exportJson.php?table=minedItemSummary crashes due to OOM.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-26T13:47:30Z

Daveh: /* Post Upgrade Issues */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.
:* Random crashes and coredumps of PHP 7.4 FPM. Increased memory limit and max files for OpCache for all PHP versions.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-24T17:38:50Z

Daveh: /* Post Upgrade Issues */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.
:* Fixed a bunch of directories with no global read/execute permissions on ''content1'' in ''/home/uesp/www''.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Administrator Noticeboard

2026-04-23T17:43:44Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers. Sites/services will be changed and tested one-by-one and listed below once verified. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 12:55, 21 April 2026 (UTC)
::* Starfield Wiki
::* UESP Wiki
::* Gamemap
::* Fallout Wiki
::* ESO Log and related services
::* ESO API/Data
::* ESO Files
::* UESP Forums
::* CSList
::* All Construction Set related Wikis
::* UESP Blog

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

UESPWiki:Administrator Noticeboard

2026-04-23T16:23:09Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers. Sites/services will be changed and tested one-by-one and listed below once verified. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 12:55, 21 April 2026 (UTC)
::* Starfield Wiki
::* UESP Wiki
::* Gamemap
::* Fallout Wiki
::* ESO Log and related services
::* ESO API/Data
::* ESO Files
::* UESP Forums
::* CSList
::* All Construction Set related Wikis

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-23T16:04:58Z

Daveh: /* Post Upgrade Issues */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog
:* Missing ''DirectoryIndex index.php'' on new Apache (was previously in ''php.conf'').
::* Added ''DirectoryIndex index.html index.htm index.php'' in ''autoindex.conf''.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Administrator Noticeboard

2026-04-23T14:39:54Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers. Sites/services will be changed and tested one-by-one and listed below once verified. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 12:55, 21 April 2026 (UTC)
::* Starfield Wiki
::* UESP Wiki
::* Gamemap
::* Fallout Wiki
::* ESO Log and related services
::* ESO API/Data
::* ESO Files
::* UESP Forums

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

UESPWiki:Administrator Noticeboard

2026-04-23T14:33:56Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers. Sites/services will be changed and tested one-by-one and listed below once verified. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 12:55, 21 April 2026 (UTC)
::* Starfield Wiki
::* UESP Wiki
::* Gamemap
::* Fallout Wiki
::* ESO Log and related services
::* ESO API/Data
::* ESO Files

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-23T13:26:40Z

Daveh: /* Post Upgrade Issues */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.
:* Missing configs from the content2 Apache:
::* beyondskyrim
::* esodata
::* uesp-blog

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-23T00:37:17Z

Daveh: /* Post Upgrade Issues */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.
:* The ''UespMysqlMemcachedSession'' code that worked on PHP 7.1 doesn't work correctly in PHP 7.4. Code fixed to enable the ESO build editor and gamemap to save correctly.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Administrator Noticeboard

2026-04-22T19:50:14Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers. Sites/services will be changed and tested one-by-one and listed below once verified. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 12:55, 21 April 2026 (UTC)
::* Starfield Wiki
::* UESP Wiki
::* Gamemap
::* Fallout Wiki
::* ESO Log and related services

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-22T12:33:36Z

Daveh: /* Page Speed Loads */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 120 / 120 / 110 ms
::* Average = 117 ms (3.2x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-21T23:38:27Z

Daveh: /* Benchmarks */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

=== Page Speed Loads ===
The following is for the UESP Wiki:

:* Old Servers
::* Content1/2/3/4 = 230 / 210 / 240 / 700 ms
::* Average = 380 ms (not counting content3)
:* New Servers
::* Content1/2/3 = 110 / 120 / 150 ms
::* Average = 130 ms (2.9x faster)

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

File:TestEsoIcon72.png

2026-04-21T23:35:05Z

Daveh: Daveh uploaded a new version of File:TestEsoIcon72.png

== Summary ==
test
== Licensing ==
{{Cc-by-sa-2.5}}

File:TestEsoIcon72.png

2026-04-21T23:34:47Z

Daveh: Daveh uploaded a new version of File:TestEsoIcon72.png

== Summary ==
test
== Licensing ==
{{Cc-by-sa-2.5}}

File:TestEsoIcon72.png

2026-04-21T23:34:30Z

Daveh: test

== Summary ==
test
== Licensing ==
{{Cc-by-sa-2.5}}

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-21T20:13:22Z

Daveh: /* Post Upgrade Issues */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.
:* content1 missing the ''/etc/httpd/conf.d/00-cache.conf'' file.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-21T20:04:49Z

Daveh: /* Post Upgrade Issues */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.
:* Transferring the ElasticSearch indexes from old to new search1 had some issues:
::* Starfield and Fallout wiki indexes needed to be rebuilt.
::* UESP IT index replica was unassigned (?). Needs to be rebuilt.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-21T19:21:58Z

Daveh: /* Backup1 */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

==== Post Upgrade Issues ====
:* ''httpd.conf'' was left as the default config on content3 pointing to ''/var/www/'' instead of ''/home/uesp/www/'' which broke some configs which assumed the home directory was previously set to UESP.
:* All content servers had left the original PHP-FPM ports as 900X instead of the new values on non-wiki hosts.
:* Missing the ''00-php.ini'' file for all PHP versions on all content servers.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-21T15:07:33Z

Daveh: /* Setup/Design ToDo List and Questions */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
:* '''PHP FPM Ports'''
::* 9000 = PHP 7.1 (change to 9074 as there is no PHP 7.1 on the new servers)
::* 9003 = PHP 7.4 (change to 9074)
::* 9004 = PHP 8.0 (change to 9080)
::* 9005 = PHP 8.1 (change to 9081)

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Administrator Noticeboard

2026-04-21T15:03:01Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers. Sites/services will be changed and tested one-by-one and listed below once verified. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 12:55, 21 April 2026 (UTC)
::* Starfield Wiki
::* UESP Wiki
::* Gamemap

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

UESPWiki:Administrator Noticeboard

2026-04-21T13:38:20Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers. Sites/services will be changed and tested one-by-one and listed below once verified. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 12:55, 21 April 2026 (UTC)
::* Starfield Wiki

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

UESPWiki:Administrator Noticeboard

2026-04-21T12:55:53Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers. Sites/services will be changed and tested one-by-one and listed below once verified. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 12:55, 21 April 2026 (UTC)

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

UESPWiki:Administrator Noticeboard

2026-04-21T12:55:44Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers. Sites/services will be changed and tested one-by-one and listed below once verified.

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

UESPWiki:Administrator Noticeboard

2026-04-21T12:55:27Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)
:* '''Cache/Content -- 21-22 April 2026''' -- We will be switching all cache and content servers to the cluster over the next few days. This will be done via DNS so there will be no service interruption expected for the main sites. For secondary sites/services we may need upgrade or fix things if they don't work properly on the new servers.
:: Sites/services will be changed and tested one-by-one and listed below once verified.

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

User:Daveh

2026-04-20T16:09:13Z

Daveh:

*'''Real Name:''' Dave Humphrey
*'''Who Am I:''' Original founder and admin of the UESP since its creation in 1995 (and the Daggerfall text FAQ before that)
*'''Where Am I:''' Currently living and working in the Toronto, Canada area
*'''What I Do:''' Full time employee for the UESP
*'''Current Interests:''' Besides the usual ES stuff I've been lately on a quest to build the [http://www.viud.net world's toughest USB drive (www.viud.net)]. The '''[https://store.viud.net VIUD Store]''' is now open!
*'''Age:''' Born in 1973 I am 50 at this edit
*'''What I'm Playing:''' ESO...contact me as @Reorx in the [[UESPWiki:ESO_Guild|UESP guild]]
*'''Social Media'''
::* FaceBook -- [https://www.facebook.com/dave.humphrey.3152 dave.humphrey.3152]
::* Twitter -- [https://twitter.com/Dave_EGH Dave_EGH]
::* Skype -- humphrey.dave

== Site ToDo List ==
See my ever growing todo list on [https://trello.com/b/1MBWJkfF/main-board Trello]. Also see the [[UESPWiki:Task List]] page for general tasks which can be done by any editor.

== Ideas/Notes ==
Just a place for me to jot down some general ideas:
:* '''Mod Ideas'''
::* Faction with semi-random quests (ala Daggerfall), many medium-large pre-randomly generated dungeons
::* In-game strategy (ala Civilization). Build 'cities' within the game, gather resources to create buildings, gain prestige to get involved with the province politics (quest line).
::* Luggage (ala Pratchett's Disc World). A large trunk follows you around with a mind, and set of sharp teeth, of its own.

== Handy MediaWiki Queries ==

:* '''Count of Daily Edits'''
SELECT COUNT( * ) , LEFT( rev_timestamp, 8 ) AS revdate
FROM `revision`
GROUP BY revdate
:: Add a ''WHERE rev_minor_edit=0'' to not count minor edits.

:* '''Count of Daily Edits from a Given Date by User'''
SELECT COUNT(*) as cnt, rev_user, rev_user_text, LEFT(rev_timestamp, 8) as revdate
FROM revision
WHERE rev_timestamp > "20121101"
GROUP BY rev_user
ORDER by cnt;

== Useful Commands ==
:* '''List Processes Using Port 80'''
fuser 80/tcp
:* '''Memory Usage'''
free -m
top
:* '''Disk Usage'''
df
du
:* '''Group Apache Log Errors'''
egrep "\[.*\] \[error\] \[client .*\]" error_log | gawk -F ']' '{print $4}' | sort | uniq -c | sort > errors.txt
:* '''Sorting the Apache Error Log'''
awk -vFS=']' '/\[.*\] \[error\] \[client .*\]/ { a[$4]++; } END { for (i in a) print a[i], i; }' < error_log | sort -n > errors.txt
:* '''Extract Database from Full Dump File'''
sed -n '/^-- Current Database: `dbname`/,/^-- Current Database: `/p' alldbs.sql > dbname.sql
:* '''Extract Pages from Squid Log and Purge'''
awk '{print $7}' access.log > allpages.txt
grep "/wiki/" allpages.txt > mainpages.txt
grep "www.uesp.net" mainpages.txt > wwwpages.txt
egrep "m.uesp.net|mobile.uesp.net" mainpages.txt > mobilepages.txt
sort wwwpages.txt > wwwpagesort.txt
uniq wwwpagesort.txt > wwwpagesuniq.txt
php purgeList.php --purge < wwwpagesuniq.txt
:* '''Use yum EPEL Repo'''
yum install epel-release
:* '''ElasticSearch'''
::* Note that the following commands must be run either on search1 or within the cluster (replace localhost with search1.uesp.net) as ElasticSearch ports are blocked by the firewall/router.
curl localhost:9201
curl localhost:9205/_cluster/health?pretty
curl localhost:9205/_aliases?pretty
curl 'localhost:9200/_cat/indices?v'
curl localhost:9205/_stats?pretty
curl -X DELETE "localhost:9200/my-index-000001?pretty"
curl -X PUT "localhost:9200/my-data-stream/_alias/my-alias?pretty"
curl -X DELETE "localhost:9200/my-data-stream/_alias/my-alias?pretty"
::* Copies documents from a source to a destination index.
curl -X POST "localhost:9200/_reindex?pretty" -H 'Content-Type: application/json' -d'
{
"source": {
"index": "twitter"
},
"dest": {
"index": "new_twitter"
}
}'
::* Find documents with specific property value:
curl -X GET "localhost:9205/uesp_net_wiki5_content_first/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"query_string": {
"query": "value",
"default_field": "field"
}
}
}
'

::* Delete document by ID
curl -X DELETE "localhost:9205/uesp_net_wiki5_content_first/page/101976?pretty"
:* '''Updating Firewall with IP Change:'''
Login with ssh (admin@10.12.222.11)
enable 15
configure terminal
http #.#.#.# 255.255.255.255 outside

== Custom Scripts ==
:* '''[[User:Daveh/countips.sh|countips]]'''
::* Counts all connections per IP address and displays sorted list. Add an optional connection type parameter (TIME_WAIT, ESTABLISHED, LISTEN, etc...) to only list connections in that state.
:* '''[[User:Daveh/uesp-banip.sh|uesp-banip]]'''
::* Helper script for banning/unbanning IP addresses using ''iptables'' at the server level.

== Setting up Eclipse for Remote Development ==
Eclipse can be setup for remote development of PHP (and anything else) directly from the UESP servers:
:* Download the base version of Eclipse
:* Install/extract to a desired directory and run
:* Go to Help--Install New Software
:* Select ''All Sites'' from the drop down list
:* Find and select the following add-ons:
::* PHP Development Tools (PDT) SDK Feature
::* JavaScript Development Tools
::* Eclipse Web Developer Tools
::* Remote System Explorer End-User Runtime
:* Install the add-ons and restart Eclipse
:* Open a ''Remote System Explorer'' perspective (or from within Window--Show View--Other)
:* Define a new connection using a SSH session to content3.uesp.net (you need to have a shell login on content3)

== MediaWiki Distribution API ==
The ExtensionDistributor API can get you download URLs, e.g. this:

https://www.mediawiki.org/wiki/Special:ApiSandbox#action=query&format=json&list=extdistbranches&formatversion=2&edbexts=WikiEditor

returns this sort of thing:
"REL1_39": "https://extdist.wmflabs.org/dist/extensions/WikiEditor-REL1_39-76d9ffd.tar.gz",
"REL1_40": "https://extdist.wmflabs.org/dist/extensions/WikiEditor-REL1_40-d330183.tar.gz",
"master": "https://extdist.wmflabs.org/dist/extensions/WikiEditor-master-17163cd.tar.gz",

== Links ==
Links I always seem to need but never remember...

'''Internal Site Links'''
:* [[MediaWiki:Monobook.css]]
:* [[MediaWiki:Common.css]]
:* [[User:Daveh/Custom Changes]]
:* [[User:Daveh/Editable_Map_Notes]]
:* [[User:Daveh/common.js]]
:* [http://esodata.uesp.net/ esodata.uesp.net -- '''ESO Global/Function Data''']

'''General Things'''
:* [http://www.netadmintools.com/art216.html Block With IPTables]
:* [http://www.dnsstuff.com DNS Stuff]
:* [http://s23.org/wikistats/ Stats on Various Mediawikis]
:* [http://www.quantcast.com/uesp.net Quantcast-UESP]
:* [http://about.digg.com/blog/rasmus-lerdorf-php-performance PHP Performance Talk] and [http://talks.php.net/show/digg/ PHP Performance Talk Slides]
:* [http://www.maatkit.org/ MaatKit MySQL Tools]
:* [http://www.visolve.com/squid/squid26/contents.php Squid Manual]
:* [http://www.deckle.co.za/squid-users-guide/Accelerator_Mode Squid Accelerator Mode]
:* [http://www.mediawiki.org/wiki/Manual:Squid_caching MediaWiki Squid Caching]
:* [http://itspice.net/cms/library-tutorials-online-books-white-papers-ebooks-free-boooks/system-administrator-checklist Server Admin Checklist]
:* [http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/Chapter+14.+Monitoring+Squid/14.2+The+Cache+Manager/ Squid: The Definitive Guide]
:* [http://stackoverflow.com/questions/1031885/mysql-optimize-table-for-all-fragmented-tables Listing Fragmented MySQL Tables]
:* [http://www.cyberciti.biz/faq/linux-unix-delete-remove-ip-address-that-denyhosts-blocked/ Remove Blocked IP from DenyHosts]
:* [http://fedoraproject.org/wiki/EPEL/FAQ#howtouse Fedora EPEL] - Extra package list for CentOS/Fedora.
:* [http://kvz.io/blog/2007/07/28/block-brute-force-attacks-with-iptables/ Block Brute Force Attacks with IPTables]
:* [http://www.centos.org/docs/4/html/rhel-rg-en-4/s1-iptables-init.html CentOS IP Tables Documentation] - Note config file at ''/etc/sysconfig/iptables-config''
:* [http://wiki.centos.org/TipsAndTricks/BecomingRoot CentOS Sudo]
:* [https://support.google.com/legal/answer/3110420?product=googleplay&rd=2 Google Legal Forms]

'''Wiki Related'''
:* [http://semantic-mediawiki.org/wiki/Maps_examples Embedded Google Maps]
:* [https://noc.wikimedia.org/conf/ Wikipedia Configuration Files]
:* [http://stackoverflow.com/questions/26734777/yum-error-cannot-retrieve-metalink-for-repository-epel-please-verify-its-path EPEL Fix for CentOS 6.3]

'''Server Architecture:'''
:* [http://ontwik.com/python/disqus-scaling-the-world%E2%80%99s-largest-django-application/ Disqus: Scaling the World’s Largest Django Application]
:* [http://www.infoq.com/presentations/MySpace-Dan-Farino Behind the Scenes at MySpace (2009)]
:* [http://meta.wikimedia.org/wiki/Wikimedia_servers Wikipedia Servers]
:* [http://www.datacenterknowledge.com/archives/2009/10/13/facebook-now-has-30000-servers/ FaceBook Servers (2009)]
:* [http://stackoverflow.com/questions/1260134/optimizing-kohana-based-websites-for-speed-and-scalability/1283195#1283195 Optimizing Websites]
:* [http://blog.reddit.com/2010/03/and-fun-weekend-was-had-by-all.html Reddit Server Architecture (2010)]
:* [http://highscalability.com/digg-architecture Digg Server Architecture (2009)]
:* [https://serverfault.com/questions/904304/could-not-resolve-host-mirrorlist-centos-org-centos-7 Yum Repo Update]

'''Runescape Wiki'''
:* [https://meta.weirdgloop.org/w/Sub4 Sub4 Architecture Notes]
:* [https://meta.weirdgloop.org/w/Forum:Mid-2023_business_update 2023 Business Update]
:* [https://docs.google.com/document/d/1i2CF5EAA4Xh3ZWtOsglFVIizMBwWdb1vQGcr3Ijr6mw/edit#heading=h.hhevn0icya3z Copy of Moving Weird Gloop to GKE Standard]
:* [https://gerrit.wikimedia.org/g/mediawiki/services/jobrunner/ WMF Jobrunner]
:* [https://www.mediawiki.org/wiki/Shellbox WMF Shellbox]

[[Category:Test Cell Images]]

testjhgfdsa119955
estjhgfdsa
jhgfdsa119955
test edit

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-20T13:55:55Z

Daveh: /* Switchover Plan */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db1/Db2'''
::* All new servers are already set to use db2 for reads.
::* Need to do both at the same time due to how MediaWiki tries to handle replication lag (doesn't work with a daisy-chain setup).
::* Set primary wikis to read only and turn off a few specific services:
:::* Wiki jobs (probably safer to turn off)?
:::* ESO log parser
:::* XKI tracking data service
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
::* Turn off read-only mode and restore disabled services.
::* Test reads and writes.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Administrator Noticeboard

2026-04-20T13:25:22Z

Daveh: /* Server Upgrade */

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)
::* This switch was reverted due to an issue with MediaWiki replication detection.
:* '''Db1/2 -- 20 April 2026 12PM EST''' -- We will be switching both db1/db2 database servers to the new cluster sometime today. There will be a short time where all wikis will be set read-only during the transition (~5 minutes). -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:25, 20 April 2026 (UTC)

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.

User:Daveh

2026-04-19T23:28:49Z

Daveh:

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-19T17:03:09Z

Daveh: /* Switchover Plan */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db2'''
::* All new servers are already set to use db2 for reads.
::* If/when old servers need to use db2 would need to update the IP address in ''uespservers.secrets''.
:* '''Db1'''
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Search1
:# Db2
:# Db1
:# Cache1
:# Content1/2/3
:# Dev1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Server Upgrade/Cluster Upgrade 2026

2026-04-19T17:02:52Z

Daveh: /* Switchover Plan */

This is the planning and log of the hardware cluster upgrade performed in March/April 2026.

== Planning / Notes ==

=== File/Image Server ===
Notes for possibly changing/improving the file server design.

==== Current Design ====
:* Single file server running lighttpd with a CloudFlare caching/CDN (average cache HIT rate of 76%).
:* File directories are shared with content servers via NFS. Single directory per site:
/shared/uesp
/shared/sfwiki
/shared/fallout
...
:* PRO -- Simple setup that is easy to use and understand.
:* CON -- NFS share per site makes scaling more difficult. Change to a single share with sites underneath that.
:* CON -- Scaling to more than one server for redundancy is not possible in this setup.

==== Potential Designs ====
* '''Varnish Server'''
::* Redundancy of a files server could be obtained by putting files1/files2/... under Varnish for load balancing purposes.
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''CloudFlare'''
::* Redundancy/load balancing for multiple file servers could be performed using CloudFlare. (DETAILS NEEDED HERE)
::* Would still need need a way for file mirroring between multiple servers (see lysncd below).
* '''lsyncd'''
::* [https://github.com/lsyncd/lsyncd Lsyncd] Lsyncd is a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.
::* Was using/testing this to sync files between files1 and search1 for a while. Relatively easy to setup and use.
* '''GlusterFS'''
::* [https://www.gluster.org/ GlusterFS] is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming.
::* Requires a dedicated virtual disk for the shared file system.

=== Database Replication ===
Notes for replicating database from the old cluster to the new clusters for migration.
:* Open MySQL port 3306 on both firewalls between clusters. Test that you can connect back and forth.
:* Decide which database server to make the backup from. Easiest would be backup1 as no service (besides Zabbix) depends on it but would need to remove. Db2 might be more "accurate" but would require all services to switch to db1 during the backup.
:* Stop slave on backup1. Note master read file and position.
:* Backup all databases on backup1 except the following (NEEDS VERIFICATION):
::* zabbix
::* mysql
::* percona
::* information_schema
::* test
::* testwiki
::* maptest
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev
mysqldump -u root -p XXXXXX --databases DB1 DB2 .... > backup.sql
:* Temporaily change database settings for faster restores if needed (reset after restore):
innodb_file_per_table=1
skip-name-resolve
key_buffer_size=8GB
innodb_buffer_pool_size=32GB
innodb_log_buffer_size=16MB
innodb_log_file_size=200MB
innodb_write_io_threads=4
innodb_flush_method = nosync
innodb_flush_log_at_trx_commit = 2
:* Copy backup to new db1/db2/backup1 and restore.
:* Start mirroring on new db1 from old db1.
:* Start mirroring on new db2/backup1 from new db1.
:* Copy Users and Permissions
::* Can export with ``mysqldump -h db1.uesp.net -u backup -p mysql --extended-insert=FALSE --tables user db > users.sql`` but this drops the tables and recreates. Removing the drops doesn't work as new tables have different columns.
::* Can use ``pt-show-grants`` from Percona Toolkit to export users/grants but need to wait until databases are restored in order to use the script.
::* Is there a better/easier way in 2026?
:* Modify settings
::* ```default-storage-engine=myisam```
::* Modify open files limit if needed.
:* Some databases from backup1 need to be moved to the new backup1 server (or dev server): (NEEDS VERIFICATION):
::* zabbix
::* uesp_net_wikidev
::* uesp_net_wikidev_ar
::* uesp_net_wikidev_fr
::* uesp_net_wikidev_it
::* uesp_net_wikidev_pt
::* falloutde_dev
::* falloutwiki_dev
::* falloutwiki_dev_shared
::* sfwiki_dev

=== Basic Per Server Plan ===
Basic steps for migrating servers.
:* Clone [https://github.com/uesp/uesp-setup uesp-setup] repo to ''/uesp-setup'':
git clone https://github.com/uesp/uesp-setup
:* Run ''uesp-setup-common'' (note and address any installation errors):
cd /uesp-setup
./uesp-setup-common
:* Backup files on the original server as root:
su
mkdir /home/backup/migration2026
cd /home/backup/migration2026
tar --exclude='home/backup/migration2026' -zcvf home.tar.gz -C /home/ .
tar -zcvf root.tar.gz -C /root/ .
tar -zcvf etc.tar.gz -C /etc/ .
:* Copy files from the original server (on some servers this might not work if there isn't enough disk space):
mkdir /home/backup/migration2026
cd /home/backup/migration2026
rsync -avz user@oldserver.uesp.net:/home/backup/migration2026/* ./
:* Extract files into new server (probably only extract /home and /root and do config files in /etc only as needed):
cd /home/backup/migration2026
mkdir etc
tar -xzvf home.tar.gz -C /home/
tar -xzvf root.tar.gz -C /root/
tar -xzvf etc.tar.gz -C ./etc/
:* Do server specific setup either from ''uesp-setup'' scripts or manually.
::* For content servers run ''./uesp-setup-content''
:* Copy program configs as needed from ''/home/backup/migration2026/etc/''.
:* Copy cron configs as needed from ''/home/backup/migration2026/etc/cron*''.
:* Copy logrotate configs as needed from ''/home/backup/migration2026/etc/logrotate*''.
:* Check for any old ''init.d'' startup scripts in ''/home/backup/migration2026/etc/init.d/'' and change to service startup scripts as needed.

=== Special Sites/Domains/Services ===
There are a few old or special sites/domains/services that need to be noted for the migration:

:* '''EQWiki (eqwiki.net)'''
::* Dave's old EverQuest wiki. Sorely out of date (+11 years) and probably not worth keeping up. Would require a significant bump in MW upgrades.
::* Currently only hosted on content1.
:* '''DaveWiki (dave.uesp.net)'''
::* Dave's private wiki for notes, ideas and other things. Needs a significant bump in MW upgrades.
::* Currently only hosted on content3?
:* '''XKI Tracking (tracking.insight.com)'''
::* A private (login only) site for tracking pipe inspection vehicles for Dave's day job. Temporarily hosted for no ETA for moving at this time.
::* Currently only hosted on content3.
::* Uses the ```xkitracking``` database.
::* Requires the ``xki-dataupdater`` and ``xkitesttcp.php`` services to be running.
::* Can remove the ``xkidevtracking.uesp.net`` and ``xkitracking.uesp.net`` domains.
::* Uses external DNS which needs to be manually requested for a change.
:* '''Beyond Skyrim (wiki.beyondskyrim.org)'''
::* We host the main and dev wikis for the Beyond Skyrim site.
::* Will need to externally notify the Beyond Skyrim staff in order to update the DNS.
::* Currently only hosted on content2.
:* '''UESP Blog (blog.uesp.net)'''
::* WordPress site for the UESP blog. Likely needs a WP upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content2.
:* '''UESP Forums (forums.uesp.net)'''
::* Old UESP forum site running PhpBB. Needs an upgrade to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''Oblivion Character Planner (ocp.uesp.net)'''
::* A hosted fan site for planning Oblivion characters. Likely needs some code upgrades to be compatible with PHP 7.4.
::* Currently only hosted on content1.
:* '''VIUD (Virtually Indestructible USB Drive) (viud.net)'''
::* Blog for Dave's USB project. Currently just forwards to a Google blog and Shopify site.
:* '''Misc Sites'''
::* '''asayre.uesp.net''' -- Old site for an ESO user. Can be removed. Hosted on content3.
::* '''blades.uesp.net''' -- Old site for Blades beta wiki. Can be removed. Hosted on content3.
::* '''castles.uesp.net''' -- Old site for Castles beta wiki. Can be removed. Hosted on content3.
::* '''chat.uesp.net''' -- Old site for UESP chat app. Can be removed. Hosted on content3.
::* '''destiny2.uesp.net''' -- Was used for collaboration with Deltia's Gaming site. Can be removed. Hosted on content3.
::* '''dhack.uesp.net / dungeonhack.uesp.net''' -- Old site for the DungeonHack Daggerfall clone wiki. Can be removed. Hosted on content1.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net''' -- These might have been an old MineCraft server. Can be removed.
::* '''infrasight.uesp.net''' -- Temporary/old site for XK tracking. Can be removed. Hosted on content3.
::* '''obquestmod.uesp.net''' -- Temporary wiki for transferring the OB Quest Mod wiki into UESP. Can be removed. Hosted on content3.
::* '''questlist.uesp.net''' -- Temporary wiki for transferring the quest list wiki into UESP. Can be removed. Hosted on content3.
::* '''sfguides.uesp.net''' -- Temporary Starfield domain. Can be removed. Hosted on content1.
::* '''sfwiki.uesp.net''' -- Temporary Starfield domain. Domain can be removed. Hosted on content3.
::* '''starfieldbuilder.com / starfieldguides.com''' -- Old planned sites that never happened. Can be removed (forward to SF wiki).
::* '''teamspeak.uesp.net / ts.uesp.net''' -- UESP TeamSpeak server. Hosted on content3.
::* '''thal-j.uesp.net / thalj.uesp.net''' -- Dev sites for TJ, front end dev.

=== Setup/Design ToDo List and Questions ===
:* '''Move dev or test specific databases on backup1 to dev. Keep backup1 database a strict replication from db1.'''
::* Yes?
:* '''Any changes to the files1 setup?'''
::* Not at this time. We will setup ''lsyncd'' on files1 to backup1/files2 for file replication.
:* '''Is there still a need for nginx on squid1?'''
::* Yes (Varnish still does not support SSL connections)
:* '''Rename squid1 to cache1.'''
::* Yes
:* '''Minimum PHP level to install on new servers?'''
::* Try to upgrade everything to use PHP 7.4 or above.
:* '''Automatic SSL Setup for Domains'''
::* DETAILS NEEDED
:* '''Remove Old/Unused Accounts'''
::* sshbackup
::* uespinfo
::* svn
::* qmail
::* alias
::* qmaild
::* qmaill
::* qmailp
::* qmailg
::* qmailr
::* qmails
::* vpopmail
::* opendkim
::* techiweb
::* lswadmin
::* sshadmin
:* '''Remove Old User Accounts (Keep User Backups)'''
::* rpeh
::* jakatackka
::* kiz
::* dania
::* thelab
::* sfguides
::* avron
::* steve
::* jeremy
:* '''Only Copy Login Passwords for the Following Users'''
::* dave
::* eckserah
::* thal-j
::* robinhood
:* '''Remove Old Domains/Sites'''
::* '''starfieldbuilder.com / starfieldguides.com '''
:::* Set to forward to ''starfieldwiki.net'' on CloudFlare page rules.
:::* Removed Apache config on content1.
:::* Added domains to ''sfwiki.conf'' Apache config.
::* '''sfguides.uesp.net / sfwiki.uesp.net'''
:::* Removed subdomains from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''questlist.uesp.net / obquestmod.uesp.net '''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''infrasight.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''dhack.uesp.net / dungeonhack.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content1.
::* '''destiny2.uesp.net'''
:::* Removed subdomain from Cloudflare DNS.
::* '''chat.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
::* '''castles.uesp.net / blades.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Disabled Apache config on content3.
::* '''asayre.uesp.net'''
:::* Removed subdomain from Cloudflare DNS. Is in ''banned.conf'' on content3.
:* '''Upgrade Old Sites'''
::* '''VIUD Blog'''
:::* Was only forwarding to the Google blog on content3. Remove content3 Apache config and change forwarding to be done within CloudFlare.
::* '''Oblivion Character Planner'''
:::* Currently running on content1 using PHP 5.6.
:::* Doesn't actually use any PHP (all client side JS).
:::* Changed to use PHP 7.4.
::* '''UESP Forums'''
:::* Currently running phpBB 3.0.12 on content2 using PHP 5.6.
:::* phpBB 3.0.12 is compatible with PHP versions up to 5.6.x. It is not compatible with PHP 7.0 or newer without significant code changes.
:::* phpBB 3.3.x is compatible to PHP 7.2.0 to 8.3 (and maybe 8.4 in later updates).
:::* [https://www.phpbb.com/support/docs/en/3.3/ug/upgradeguide/upgrade3/ Upgrading phpBB from 3.0 to 3.3]
:::* Updated to 3.3.14 running PHP 7.4. Manually edited the ''prosilver'' style with images/colors to match the UESP theme.
::* '''UESP Blog'''
:::* Currently running WordPress 5.7.14 and News Vibrant Pro theme. Using PHP 8.0.28 on content2.
:::* WordPress 5.7.14 can function with PHP versions from 5.6.20+ up to 8.2.x.
::* '''DaveWiki'''
:::* Upgraded to MW 1.35 using PHP 7.4.
::* '''EQWiki'''
:::* Was already set to page reads via login. Removed config from content1 Apache. Added to ''banned.conf''.
::* '''Other Sites'''
:::* Set TamrielRebuilt map to use PHP 7.4.
:* '''Add TLS 1.3 Support to Apache'''
::* Requires Apache 2.4.37 or above. Changed ``SSLProtocol`` on content4 to:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 +TLSv1.3
::* Nginx version 1.13.0 or greater required, version 1.20+ is recommended (current version on squid1 is 1.16.1). OpenSSL version 1.1.1 or greater.
::* Use the recommended protocol (currently set to this):
ssl_protocols TLSv1.2 TLSv1.3;
:* '''DNS Settings'''
::* Create subdomains for old servers in the form ''old-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).
::* Create subdomains for new servers in the form ''new-content1.uesp.net'', non-proxied for now (unless there are bot issues or once the new cluster goes live).

== Upgrade Log ==
Record basic operations, notes, and any issues/resolutions on the upgrade process here.

=== Pre-Upgrade Log ===
:* Removed old/unused sites from Apache config on content servers.
::* starfieldbuilder.com / starfieldguides.com
::* sfguides.uesp.net / sfwiki.uesp.net
::* questlist.uesp.net / obquestmod.uesp.net
::* fallenrp.uesp.net / gsg.uesp.net / maxping.uesp.net / mc.uesp.net / minecraft.uesp.net / nebula.uesp.net
::* infrasight.uesp.net
::* dhack.uesp.net / dungeonhack.uesp.net
::* destiny2.uesp.net
::* chat.uesp.net
::* castles.uesp.net / blades.uesp.net
::* asayre.uesp.net
::* eqwiki.net
:* Upgraded sites still using PHP 5.6
::* VIUD Blog
::* Oblivion Character Planner
::* UESP Forums
::* UESP Blog
::* DaveWiki
:* Added DNS entries for all ''old-'' servers.
:* Setup CF Tunnel on old servers for testing:
curl -fsSl https://pkg.cloudflare.com/cloudflared.repo | sudo tee /etc/yum.repos.d/cloudflared.repo
yum install cloudflared
cloudflared service install <token>
::: On the Windows client install/run:
https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-windows-amd64.msi
cloudflared.exe service install <token>
::: This works on content4 but not content3 assumably due to the older CentOS distribution.

=== New Server Log ===

====Overall Status====
:* '''Content1 --''' Basic content setup, file migration started.
:* '''Content2 --''' Basic content setup, file migration started.
:* '''Content3 --''' Basic content setup, file migration started.
:* '''Dev1 --''' Basic content installed, file migration started, Apache running need to configure sites and service.
:* '''Db1 --''' Common setup done. Replication started from old db1. Files copied.
:* '''Db2 --''' Common setup done. Replication started from new db1. Files copied.
:* '''Cache1 --''' Common setup done, varnish and nginx installed.
:* '''Files1 (Switched) --''' Everything done and tested, ready for migration (just need the /shared files updated from old files1).
:* '''Search1 --''' Common setup done.
:* '''Backup1 --''' Zabbix server and agent installation "done". MariaDB server installed for Zabbix. Replication started from new db1. Files copied.
:* '''Filewall --''' Setup old/new crosstalk done.

==== General ====
:* Added DNS entries for all old- and new- servers.
:* Install ''git'' on the server.
dnf install git
:* Clone the ''uesp-setup'' repo into the base directory.
cd /
git clone https://github.com/uesp/uesp-setup.git
:* Run ''uesp-setup-common HOSTNAME 10.12.123.17'' (change HOSTNAME to the specific host, eg: ''content1'') and note any issues.
::* Warning about ''uesp-setup-rclocal'' (still need this for ''denyhost'' startup).
::* Added script to update default vim colorscheme to ''desert''.
:* For all servers
::* Updated ''/etc/uesp-keys'', copied PUB keys to dev1 and then to all other servers.
::* Installed salt-minion on all servers, salt-master on dev1, and setup all minions on master.
::* Updated ''/etc/hosts'' with private IPS for all servers.
:*'''Mail Setup'''
::* Create a Google app password for use with the mail.
::* Run:
dnf install sendmail sendmail-cf s-nail
mkdir -m 700 /etc/mail/authinfo/
::* Create ''/etc/mail/authinfo/gmail-auth'' with the following content:
AuthInfo: "U:root" "I:YOUR GMAIL EMAIL ADDRESS" "P:YOUR PASSWORD"
::* Edit ''/etc/mail/sendmail.mc'' with the following just above the first MAILER definition:
define(`SMART_HOST',`[smtp.gmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail-auth.db')dnl
::* Run:
makemap hash gmail-auth < gmail-auth
make -C /etc/mail
systemctl enable sendmail
systemctl restart sendmail
::* Test with:
echo "Just testing my sendmail gmail relay" | mail -s "Sendmail gmail Relay" my-email@my-domain.com
:* Added ''AllowKey=system.run[*]'' to ''/etc/zabbix/zabbix_agent2.conf'' on content1/2/3 and dev1 for the PageSpeed template.
:* Permitted global reads on ''/var/log/httpd/'' to permit the PageSpeed template to work.
:* Updated sudoers on all servers.

==== Firewall ====
:* Added Old Servers with MySQL/NFS/Memcached services to the new firewall.
:* Added New Servers with MySQL/NFS/Memcached services to the old firewall.
:* Turned off NAT for all new firewall rules so that the source IP is correctly transmitted.
:* Added Old Servers with ElasticSearch services to the new firewall.

==== Content1 ====
:* '''Dave''' : Ran ''uesp-setup-common''
::* This was the first server setup so a bunch of issues with the script were fixed and improved.

:* '''Eckserah''' : Installed cloudflared for allowing ssh access through the cloudflare proxy.
::* Working currently on the local changes needed to allow connection on my machine. (put on hold)
::* Ran ''uesp-setup-content'' : Didn't run into any issues after running.
::* Started the file migration

:* '''Dave''' : Setting up the content apps:
::* php-pecl-memcached not installed due to a conflict. Unsure if we need the memcached or memcache library here for MW (or either).
::* mod_evasive: On AlmaLinux 9 you have to compile/install from source (see https://github.com/shivaas/mod_evasive).
::* Reran ''uesp-update-hostname content1''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah''' : Finished file migration of home and root directories.
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Tested SSL with SSL Labs (no issues noted).

==== Content2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
:* '''Dave'''
::* Reran ''uesp-update-hostname content2''
::* Ran ''uesp-setup-mounts''
:* '''Eckesrah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
:* '''Dave'''
::* Disabled mod_evasive.
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.

==== Content3 ====
:* Note that the new content3 is the old content4

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration

:* '''Dave'''
::* Reran ''uesp-update-hostname content3''
::* Ran ''uesp-setup-mounts''

:* '''Eckserah'''
::* Copied etc directory for use on case by case basis.
::* Set up symbolic link to certbot certificates. <code>ln -s /mnt/shared/certbot /etc/letsencrypt</code>
::* Updated configs to point to the correct new servers.
::* Finished file migration. In order to get everything to transfer I had to use --bwlimit=10M to limit the file transfer. 10Mb/s seems to be the best I could do in order to not cause content4 to get overloaded.

:* '''Dave'''
::* Disabled mod_evasive (was causing issues with loading wiki with 403s).
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server.
::* Added missing UESP section to ''/etc/httpd/conf/httpd.conf'' to match other content servers.

==== Dev1 ====
:* Note that the new dev1 is the old content3

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Ran ''uesp-setup-content'' : Encountered no issues.
::* Started the file migration
::* Apache installed and running. Need to set up services.
::* Services and sites set up. Uesp and fallout confirmed working.
:::* Used config files from content4. Will need to modify the configs for sites not on content4.
::* Changed php ports to match the version numbering for ease of use.
::* Set up certbot on dev1 with auto renewing certificates. Certificates are stored in <code>/mnt/shared/certbot/live/${SiteName}/</code> Will automatically renew for starfield, fallout, and uesp. Need to get vaultpedia on cloudflare to be able to have it work automatically there too.
:::* Starfield: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/sfwiki-cloudflare.ini -d *.starfieldwiki.net -d starfieldwiki.net --cert-name starfieldwiki.net --dry-run</code>
:::* UESP: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/uesp-cloudflare.ini -d *.uesp.net -d uesp.net -d *.m.uesp.net --cert-name uesp.net --dry-run</code>
:::* Fallout: <code>certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/falloutwiki-cloudflare.ini -d *.fallout.wiki -d fallout.wiki --cert-name fallout.wiki --dry-run</code>
:::* Additionally set up ssl-shared.common which will allow all our sites to pull from the correct paths. Uses an enviorment variable "sslPath" which is the name of the folder (Folders are named for the site url e.g: fallout.wiki, uesp.net, vaultpedia.de, starfieldwiki.net).
::* Set up services that were already set up on the old content3.
::* Set up cron jobs and logrotate that was already on old content3 (cron jobs disabled temporarily until it's time to move everything over).
::* '''TODO''': Install database and copy over.

==== Db1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave'''
::* Fixed hostname via ''uesp-update-hostname db1''.
::* Fixed and reran ''uesp-fix-homeperms''..
::* Installed Zabbix Agent2 via ''uesp-install-zabbixagent db1 10.12.123.17'' and added to Zabbix Server.
::* Setup MariaDB Server
:::* ''yum install mariadb-server''
:::* ''systemctl enable mariadb''
:::* ''systemctl start mariadb''
:::* ''mysql_secure_installation''
:::* ''mkdir /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4001.
:::* Restarted MariaDB.
:::* Checked file limit for MariaDB: 50005 / 50005 (limited to 5000 by server config).
:::* Changed some server settings:
::::* key_buffer_size = 24G
::::* max_allowed_packet = 200M
::::* sort_buffer_size = 1M
::::* myisam_sort_buffer_size = 2G
::::* tmp_table_size = 200M
::::* innodb_buffer_pool_size = 90G
::::* innodb_log_buffer_size = 32M
:::* Temporary settings for data import:
::::* innodb_flush_log_at_trx_commit = 2
::::* innodb_flush_method = nosync
:::* Install Percona Toolkit
::::* yum install perl-sigtrap perl-English perl-FindBin perl-Time-HiRes perl-DBD-MySQL
::::* rpm -ivh https://downloads.percona.com/downloads/percona-toolkit/3.7.1/binary/redhat/9/x86_64/percona-toolkit-3.7.1-2.el9.x86_64.rpm
::* Copied latest alldbs backup from old backup1 with master.txt info.
::* Started database restoration.
:::* Needed to add SHOW VIEW, EVENT, and TRIGGER permissions to backup user on old backup1 (also added on old db1/db2).
:::* Full dump on backup1 took 6.8 hours
real 406m40.841s
user 98m53.214s
sys 12m52.733s
:::* 563GB uncompressed size
:::* 8.9 hours to restore
real 532m48.896s
user 76m44.723s
sys 10m23.869s
:::* Note that this database backup contains some of the backup1 specific databases as well which will need deleting.
::::* falloutde_dev
::::* falloutwiki_dev
::::* falloutwiki_dev_shared
::::* sfwiki_dev
::::* uesp_net_wikidev_ar (also on old db1)
::::* uesp_net_wikidev_fr (also on old db1)
::::* uesp_net_wikidev_it (also on old db1)
::::* uesp_net_wikidev_pt (also on old db1)
::::* '''Note''': ''beyondskyrim_devwiki'' is the developer wiki for Beyond Skyrim and should be kept on db1.
:::* Ran ''pt-show-grants'' on old db1 and copied over to new db1. Removed a few entries (root), fixed a few entries that had no password set (needs a password to create user), and a few entries in the wrong order (GRANT permissions before the password was set).
:::* Started replication from old db1 to new db1:
Relay_Master_Log_File: uesp-mysql-bin.006100
Exec_Master_Log_Pos: 435085652
File: uesp-mysql-bin.000708
Position: 869681628
::* Copied /home and /root to new server (deleted a few old things from /home/backup).
::* Started replication from new db1 to backup2.
::* Removed temporary import settings from MariaDB config and restarted.

==== Db2 ====
:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.
::* Added REPLICA MASTER privilege to wikiuser
:* '''Dave'''
::* Fixed hostname via uesp-update-hostname db1.
::* Fixed and reran uesp-fix-homeperms..
::* Installed Zabbix Agent2 via uesp-install-zabbixagent db2 10.12.123.17 and added to Zabbix Server.
::* Setup MariaDB (same as for db1).
::* Replication started from new-db1.
::* Reverted temporary import settings for MariaDB and restarted.

==== Cache1 ====
:* Formerly known as squid1
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
::* Installed varnish and nginx
::* Fixed the varnish and nginx configs.
::* Added symbolic link to the certbot certificates.
::* Sites accessible through new-cache1.uesp.net, new-cache1.fallout.wiki, new-cache1.starfieldwiki.net
:* '''Dave'''
::* Uninstalled Zabbix Agent and installed Zabbix Agent2 and setup with Zabbix server. Note that the ''yum'' install procedure decided to upgrate ''nginx'' in the process.
::* Ran ''usermod -aG varnish zabbix'' to permit the zabbix agent access to ''varnishstat''. Restarted Zabbix Agent2.
::* Imported the Zabbix Varnish template and modified entries parsed from ''varnishstat'' output to include the ''counters'' top level array in JSON data.
::* Fixed the Nginx Zabbix by modifying the template macro to point to ''https://localhost/basic_status'' for the get stub status page.

==== Files1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''': When setting up the file share with ''uesp-setup-shares'' found the following issues. These appear to be old services no longer needed.
Failed to enable unit: Unit file nfs-lock.service does not exist.
Failed to enable unit: Unit file nfs-idmap.service does not exist.
Failed to start nfs-lock.service: Unit nfs-lock.service not found.
Failed to start nfs-idmap.service: Unit nfs-idmap.service not found.
::* Tested file sharing with new content1 and old content3 successfully.
::* Note that the new sharing setup will share ''files:/shared/'' as ''content1:/mnt/shared/''. Make symbolic links for the existing ''/mnt/uesp/'' directories to ''/mnt/shared/uesp/''
::* Modified the ''uesp-setup-mounts'' script to do this for the new servers.
::* Copied the ''/shared/'' folder from old files1.
::* Installed lighttpd / php-fpm via ''uesp-install-lighttpd''
:::* Replaced mod_compress with mod_deflate
:::* Added mod_authn_file
:::* Installed missing lighttpd-fastcgi
:::* Note: ssl.use-sslv2 is deprecated
:::* Note: ssl.use-sslv3 is deprecated
:::* Started lighttpd and php-fpm
::* Basic benchmarking test for images.uesp.net/6/6a/SR-cover-Skyrim_Box_Art.jpg (475 kb)
:::* x1.34 faster on new-files1 (244 req/sec).
::* Basic bencharmking test for images.uesp.net/thumb/6/6a/SR-cover-Skyrim_Box_Art.jpg/81px-SR-cover-Skyrim_Box_Art.jpg (1.8 kb)
:::* x57.7 faster on new-files1 (30185 req/sec)!
::* Check Files Limit
:::* System limit of 65536/65536 (8192/8192 on old files1).
:::* Limit for lighttp set to 6000/6000, for php-fpm set to 1024/524288 (same soft limits as old files1).
::* Test Dynamic Thumbnail creation
:::* Works (added temporary handling for new-files1 in images.conf).
::* Setup New Mount on Old Servers
:::* /mnt/shared setup on content1/2/3/4 and backup1
:::* Created the ''/mnt/shared/uesp-switch-oldmounts'' script to swap the old mounts to the new mounts on the old content servers. Use ''/mnt/shared/uesp-restore-oldmounts''to remount the old files1 shares.
::* Copied ''/home/'' from old files1.
::* Copied ''/root/'' from old files1.
::* Installed Zabbix Agent 7.4.
::* Removed Zabbix Agent, installed Agent 2. Added to Zabbix server on new backup1.
::* Added Lighttpd template to Zabbix and added to files1 host.
::* '''Reboot Test'''
::* Lighttpd up
::* Denyhosts up
::* Zabbix Agent up
::* Mounts ok

==== Search1 ====
:* '''Eckserah''' : Ran ''uesp-setup-common'' : Encountered no issues.
:* '''Dave''':
::* Increased ''memcached'' memory usage from 5GB to 10GB.

==== Backup1 ====
:* Note: Update zabbix-server installation script (updated the rpm but not beyond that).
:* Question: Should we install zabbix-server from scratch and not restore existing database? Or upgrade from existing?
::* Attempt to upgrade from existing and if that either becomes to complex or isn't feasible install from scratch.

:* '''Eckserah'''
::* Ran ''uesp-setup-common'' : Encountered no issues.

:* '''Dave'''
::* Reran ''uesp-update-hostname backup1''
::* Ran ''uesp-setup-mounts''
::* Installed Zabbix server/agent (server not yet setup).
::* Installed MariaDB server, started/enabled, and ran ''mysql_secure_installation''.
::* Ran ''uesp-setup-content''
::* Re-ran ''uesp-update-yum''
::* Re-ran ''uesp-install-contentapps backup1 10.12.123.17''
::* Setup and started Apache using PHP 8.0.
::* Setup Zabbix Server:
:::* Mysql: create database zabbix character set utf8mb4 collate utf8mb4_bin;
:::* Mysql: grant all privileges on zabbix.* to zabbix@localhost identified by '<password>';
:::* Mysql: set global log_bin_trust_function_creators = 1;
:::* Run: zcat /usr/share/zabbix/sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
:::* Ran the web install.
:::* Uninstalled Zabbix Agent and reinstalled Agent2.
:::* Created monitor user for Zabbix agent in MySQL
CREATE USER 'zbx_monitor'@'%' IDENTIFIED BY '...';
GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW,SLAVE MONITOR ON *.* TO 'zbx_monitor'@'%';
:::* Changed agent server to 127.0.0.1 (using 10.12.123.17 resulted in some access denied errors).
:::* Added MySQL template to backup1.
::* Setup new 8TB RAID1 HDD mount at ''/mnt/data''.
::* Started file transfer from old backup1:
:::* /mnt/data/
::* Fixed some MariaDB settings
:::* ''mkidr /mysqltmp''
:::* ''chown mysql:mysql /mysqltmp''
:::* Changed references to ''/var/log/mysql/...'' to ''/var/log/mariadb/...''.
:::* Updated server-id to 4003.
:::* Restarted MariaDB.
::* Performed same MariaDB setup as on db1.
::* Started replication from new db1.
::* Reverted temporary import settings in MariaDB and restarted.
::* Finished copying ''/mnt/data'', created symlink to ''/home2''..
::* Copied ''/home'' from old backup1.
::* Deleted some old data
:::* Old forums/blog backups.
:::* Compressed old esochardata/esobuilddata files.
::* Updated directory names in ''/home2/backup/servers/'':
:::* content3 to dev1
:::* content4 to content3
:::* squid1 to cache1
::* Copied ''/root'' from old-backup1 to ''/root/old/''.

== Benchmarks ==
Quick benchmarks for primary services:
:* New Server HTTP: ''hey -c 10 -z 30s URL''
:* Old Server HTTP: ''ab -kc 10 -t 30 URL''
:* Old ElasticSearch: ''ab -c 10 -t 30 http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* New ElasticSearch: ''hey -c 10 -z 30s http://localhost:9204/uesp_net_wiki5_general_first/_search?q=vampire''
:* Use the private IP or non-proxied domain name to avoid CloudFlare issues.
:* Note that benchmarks on live servers may be reduced due to load.

{| class="wikitable sortable"
|-
! Server !! Benchmark !! Old !! New
|-
| content1 || Main_Page || 41 req/sec || 445 req/sec
|-
| content2 || Main_Page || 33 req/sec || 428 req/sec
|-
| content3 || Main_Page || 43 req/sec || 426 req/sec
|-
| dev1 / content4 || Main_Page || 13 req/sec || 374 req/sec
|-
| squid1 / cache1 || Main_Page || 204 req/sec || 4700 req/sec
|-
| content1 || Special:RecentChanges || 18 req/sec || 48 req/sec
|-
| content2 || Special:RecentChanges || 16 req/sec || 44 req/sec
|-
| content3 || Special:RecentChanges || 23 req/sec || 44 req/sec
|-
| dev1 / content4 || Special:RecentChanges || 8 req/sec || 43 req/sec
|-
| squid1 / cache1 || Special:RecentChanges || 123 req/sec || 5525 req/sec
|-
| content1 || Online:Crown_Store/all || 26 req/sec || 214 req/sec
|-
| content2 || Online:Crown_Store/all || 25 req/sec || 208 req/sec
|-
| content3 || Online:Crown_Store/all || 33 req/sec || 201 req/sec
|-
| dev1 / content4 || Online:Crown_Store/all || 7 req/sec || 195 req/sec
|-
| squid1 / cache1 || Online:Crown_Store/all || 37 req/sec || 797 req/sec
|-
| files1 || /4/40/NewUESPLogo1.png || 713 req/sec || 9505 req/sec
|-
| search1 (from localhost) || /uesp_net_wiki5_general_first/_search?q=vampire || 6991 req/sec || 8266 req/sec
|-
| search1 (from content3) || /uesp_net_wiki5_general_first/_search?q=vampire || 4825 req/sec || 10075 req/sec
|}

== Switchover Plan ==
The following details plans for switching from old to new servers/services.

:* '''Files1 (Switched)'''
::* New servers are already set to use new files1 via NFS.
::* Update old servers to mount from the new files1 (via /mnt/shared/) and add sym links for old directories.
::* Rsync from old files1 ''/shared'' to new files1.
::* Change DNS entries to point to new files1.
:* '''Search1'''
::* Open necessary ports in old and new firewall for ElasticSearch (9202 to 9207).
::* Stop all wiki jobs.
::* Rsync ElasticSearch files from old search1 ''/home/uesp/elastic*'' to new search1.
::* Change IP address in ''uespservers.secrets'' to point to new search1.
::* Restart wiki jobs.
:* '''Db2'''
::* All new servers are already set to use db2 for reads.
::* If/when old servers need to use db2 would need to update the IP address in ''uespservers.secrets''.
:* '''Db1'''
::* Change IP address in ''uespservers.secrets'' for old and new servers to point to new db1.
::* Stop replication on new db1 from old db1.
:* '''Cache1'''
::* Should be mostly just changing DNS entries.
:* '''Content1/2/3'''
::* Should be mostly just changing DNS entries.
:* '''Dev1'''
::* Should be mostly just changing DNS entries.
:* '''Backup1'''
::* Just need to update the DNS for monitor.uesp.net.

Rough order to change services:
:# Files1
:# Db1/Db2
:# Search1
:# Content1/2/3
:# Dev1
:# Cache1

== References ==
:* [[UESPWiki:Server Upgrade/2020|Cluster Upgrade 2020]] -- Notes from the previous cluster upgrade.
:* [https://orcacore.com/set-up-nfs-server-client-almalinux-9/ NFS Setup on AlmaLinux 9]
:* [https://github.com/rakyll/hey Hey Benchmark Tool]

UESPWiki:Administrator Noticeboard

2026-04-19T14:45:18Z

Daveh: /* Server Upgrade */ db2

{{protection|move=full}}{{Trail}}[[Category:Message Boards]][[Category:UESP Administration]] __NEWSECTIONLINK__<div style="float:right">{{Scrolling Archive Table|class=collapsible collapsed|height=225px|width=225px
|UESPWiki:Administrator Noticeboard/Archive 1|Archive 1|Feb 2005 – Jun 2006
|UESPWiki:Administrator Noticeboard/Archive 2|Archive 2|Jun 2006 – Aug 2006
|UESPWiki:Administrator Noticeboard/Archive 3|Archive 3|Sep 2006 – Jun 2007
|UESPWiki:Administrator Noticeboard/Archive 4|Archive 4|Jun 2007 – Jul 2007
|UESPWiki:Administrator Noticeboard/Archive 5|Archive 5|Aug 2007 – Oct 2007
|UESPWiki:Administrator Noticeboard/Archive 6|Archive 6|Oct 2007 – Nov 2007
|UESPWiki:Administrator Noticeboard/Archive 7|Archive 7|Dec 2007 – Jan 2008
|UESPWiki:Administrator Noticeboard/Archive 8|Archive 8|Jan 2008 – Feb 2008
|UESPWiki:Administrator Noticeboard/Archive 9|Archive 9|Feb 2008 – Aug 2008
|UESPWiki:Administrator Noticeboard/Archive 10|Archive 10|Sep 2008 – Oct 2008
|UESPWiki:Administrator Noticeboard/Archive 11|Archive 11|Nov 2008 – Dec 2008
|UESPWiki:Administrator Noticeboard/Archive 12|Archive 12|Dec 2008 – Mar 2009
|UESPWiki:Administrator Noticeboard/Archive 13|Archive 13|Apr 2009 – Dec 2009
|UESPWiki:Administrator Noticeboard/Archive 14|Archive 14|Jan 2010 – May 2010
|UESPWiki:Administrator Noticeboard/Archive 15|Archive 15|Jun 2010
|UESPWiki:Administrator Noticeboard/Archive 16|Archive 16|Jun 2010 – Oct 2010
|UESPWiki:Administrator Noticeboard/Archive 17|Archive 17|Nov 2010 – Jan 2011
|UESPWiki:Administrator Noticeboard/Archive 18|Archive 18|Feb 2011 – Apr 2011
|UESPWiki:Administrator Noticeboard/Archive 19|Archive 19|May 2011
|UESPWiki:Administrator Noticeboard/Archive 20|Archive 20|Jun 2011 – Aug 2011
|UESPWiki:Administrator Noticeboard/Archive 21|Archive 21|Sep 2011 – Oct 2011
|UESPWiki:Administrator Noticeboard/Archive 22|Archive 22|Nov 2011 – Dec 2011
|UESPWiki:Administrator Noticeboard/Archive 23|Archive 23|Dec 2011 – Feb 2012
|UESPWiki:Administrator Noticeboard/Archive 24|Archive 24|Feb 2012 – Mar 2012
|UESPWiki:Administrator Noticeboard/Archive 25|Archive 25|Apr 2012
|UESPWiki:Administrator Noticeboard/Archive 26|Archive 26|May 2012 – Jun 2012
|UESPWiki:Administrator Noticeboard/Archive 27|Archive 27|Jul 2012 – Aug 2012
|UESPWiki:Administrator Noticeboard/Archive 28|Archive 28|Aug 2012 – Oct 2012
|UESPWiki:Administrator Noticeboard/Archive 29|Archive 29|Oct 2012 – Dec 2012
|UESPWiki:Administrator Noticeboard/Archive 30|Archive 30|Jan 2013 – Mar 2013
|UESPWiki:Administrator Noticeboard/Archive 31|Archive 31|Mar 2013 – Jun 2013
|UESPWiki:Administrator Noticeboard/Archive 32|Archive 32|Jul 2013 – Nov 2013
|UESPWiki:Administrator Noticeboard/Archive 33|Archive 33|Dec 2013 – Feb 2014
|UESPWiki:Administrator Noticeboard/Archive 34|Archive 34|Mar 2014 – Aug 2014
|UESPWiki:Administrator Noticeboard/Archive 35|Archive 35|Aug 2014 – Nov 2014
|UESPWiki:Administrator Noticeboard/Archive 36|Archive 36|Nov 2014 – Sep 2015
|UESPWiki:Administrator Noticeboard/Archive 37|Archive 37|Mar 2015 – Feb 2016
|UESPWiki:Administrator Noticeboard/Archive 38|Archive 38|Mar 2016 – May 2017
|UESPWiki:Administrator Noticeboard/Archive 39|Archive 39|Jun 2017 – Jul 2018
|UESPWiki:Administrator Noticeboard/Archive 40|Archive 40|Aug 2018 – Dec 2018
|UESPWiki:Administrator Noticeboard/Archive 41|Archive 41|Jan 2019 – Dec 2019
|UESPWiki:Administrator Noticeboard/Archive 42|Archive 42|Jan 2020 – Dec 2020
|UESPWiki:Administrator Noticeboard/Archive 43|Archive 43|Jan 2021 – Dec 2021
|UESPWiki:Administrator Noticeboard/Archive 44|Archive 44|Jan 2022 – Dec 2022
|UESPWiki:Administrator Noticeboard/Archive 45|Archive 45|Jan 2023 – Dec 2023
}}{{TOCright|limit=3}}</div>
The '''Administrator Noticeboard''' is where we discuss and coordinate administrative tasks on the wiki. Although this page is intended for [[UESPWiki:Administrators|administrators]], all users are encouraged to participate in discussions.

'''Before starting a discussion here''', please review the other community pages below, as your question or suggestion may be more appropriate on another page.
{{Example Table|
Other pages for community-wide or general questions include:
{{Bullet Link|[[:Category:Policies and Guidelines|Policies and Guidelines]]|A list of policies and guidelines, which each have individual pages; their talk pages are the best place to discuss these policies (see the Community Portal's [[UESPWiki:Community Portal#Active Discussions|Active Discussions]] for links to active policy discussions).}}
{{Bullet Link|Community Portal|The main discussion forum used for community-wide discussions about UESP's operations, policies, design, and improvement.}}
{{Bullet Link|Deletion Review|An area to discuss controversial page deletions and reach a consensus.}}
{{Bullet Link|Reference Desk|This is the best place for general questions about ''The Elder Scrolls'' games and lore.}}
Specific requests and notifications can be made on these pages:
{{Bullet Link|Account Pseudonymization Requests|For removing personal details and blocking further access to your own account.}}
{{Bullet Link|[[/Block Notifications|Block Notifications]]|For blockers to record their blocks, notifying administrators to potentially take further action on them.}}
{{Bullet Link|Bot Requests|This page can be used to request that one of the wiki's bots take on a task.}}
{{Bullet Link|Creation Kit Information Requests|You can request specific Creation Kit information for articles here.}}
{{Bullet Link|Image Requests|You can request specific images for articles here.}}
{{Bullet Link|New Page Requests|You can request a new page here if you were prevented from creating the page yourself.}}
{{Bullet Link|Purge Requests|If you are having problems viewing an article on UESP, the page may need to be purged. New purge requests can be made here.}}
Past discussions from the Administrator Noticeboard can be found at:
{{Bullet Link|[[/Archives|AN Archives]]|Lists all of the past discussions from the Administrator Noticeboard, including major discussions, chronological archives, and previous requests for administrator, bureaucrat, or server access.}}
Nominations to become a [[UESPWiki:Patrollers/Nominations|patroller]] or [[UESPWiki:Autopatrolled Users#Nominations|autopatrolled user]], as well as [[UESPWiki:Bots#Bot Proposals|proposals for new bots]], should be held on their respective pages. Requests for [[UESPWiki:Userspace Patrollers#Becoming a Userspace Patroller|userspace patroller]], [[UESPWiki:Page Movers|page mover]], [[UESPWiki:Interface Editors|interface editor]], [[UESPWiki:Cartographers|cartographer]], or [[UESPWiki:Blockers#Becoming a Blocker|blocker]] rights should be made on the [[UESPWiki:Role Requests|role requests page]]. Nominations for adminship or any other [[UESPWiki:User Group Rights|user group rights]] should be posted here.
}}

== Spam Protection - Don't Allow Wiki Edits by IP addresses? ==
Sorry if this has been considered before (as I suspect it has, probably more than once), but I've noticed that most vandalism/nonsense edits relate to editors without accounts and/or users not logged in. I strongly suggest - while obviously not guaranteed to solve all such issues - all pages across the wiki should be locked from editing when the editor is not logged into a (free) account. It may go against the principle of the open nature of the wiki, but creating an account is free, and the ridiculous amount of vandalism routinely cropping up is a significant drain on the admins' limited time and detrimental to the wiki regardless. Thanks for the consideration! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:55, 2 January 2024 (UTC)

: I don't think we've been struggling particularly badly with repeated vandalism of late? We have tools available to block and revert all vandalism, IP or otherwise. Reducing the pool of editors to avoid the issue seems like overcompensation to me. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:46, 2 January 2024 (UTC)

:: To your first point, we have indeed considered blocking IP edits before, a couple of times, and every time the general consensus has been what Legoless just said. There are also a lot of useful edits coming from IP addresses, not just spam and vandalism. A recent example of this is {{Pl|UESPWiki:New Page Requests|action=history|New Page Requests}}, which has a ton of edits from one particular IP. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:25, 2 January 2024 (UTC)

::: Understood - thanks for the replies and info! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

== Watchlist and Recent Changes Options/Preferences? ==
One last question - for quite a while I've been hoping for two convenience options in user preferences: hiding pages' associated "talk" pages from the "watchlist", and selecting certain namespaces to hide from the "recent changes" view (e.g., I'm not interested in listing any "Online:" page changes). I understand the watchlist itself can be used to somewhat mitigate the second point, but are either of these options actually already available, and if so, how can I use them? If they're not available, could one or both please be considered for addition to the wiki? Thanks again! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 09:38, 3 January 2024 (UTC)

: Under Recent Changes, Select one namespace: Online -> Invert selection -> Show. This will filter out ESO results. It will not hide the Online talk namespace however. I don't know of a way to exclude talk pages from RC or watchlist results. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:15, 3 January 2024 (UTC)

:: Oh, I somehow didn't look at that; thanks! But is there a way to filter out more than one namespace at the same time, or better yet, to only show "recent changes" to specific selected namespaces (I'm most interested in only the four namespaces Daggerfall, Morrowind, Oblivion, and Skyrim)? --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 10:31, 3 January 2024 (UTC)

::: This made me wonder what "Namespace: Turn on custom list" does on the RC page. The Setup link beside it goes to a preferences page unrelated to namespace filtering.

::: As a solution, check out [[User:Dillonn241/Sandbox 1|my sandbox]] and the code below. I transcluded Recent Changes and used several parameters to specify what you want. The important difference here is that with transclusions, it lets you include multiple namespaces separated by semicolons.

:::: <code><nowiki>{{Special:RecentChanges/days=90,limit=500,hidebots,namespace=104;110;116;134}}</nowiki></code>

::: See the other possible parameters on [https://www.mediawiki.org/wiki/Help:Recent_changes MediaWiki's Recent Changes help page].

::: [https://en.uesp.net/w/api.php?action=query&meta=siteinfo&siprop=namespaces This API result] shows what UESP's namespace numbers are, in case you want to customize further.

::: —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:34, 4 January 2024 (UTC)

:::: Ha, I was JUST typing about how I snooped on your sandbox page (which I saw pop up on the Recent Changes list), and figured out exactly the above out based on that URL - thanks a lot for taking time to help me with this! I'd still like a way to hide "talk" pages from the Watchlist through a preferences option as well, but if that one's not possible or planned, so be it. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:41, 4 January 2024 (UTC)

:::::I can't find anything for addressing your watchlist question. You can't transclude it in the same way, and the "Associated namespace" feature only works to ''add'' talk pages for a namespace you select. I would just recommend going into your watchlist and purging talk pages once in awhile. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:42, 4 January 2024 (UTC)

::::::Got it; thanks very much once again. If I knew how to bake you a plate of wiki cookies, I would. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 04:43, 4 January 2024 (UTC)

{{od}} It's pretty clear from the design that they at least considered the option of watching article pages and talk pages separately, but right now, having them go hand in hand is pretty much baked in. I've tried removing one but not the other from my watchlist on a test wiki, and it worked, but being so baked in, it would take a significant rewrite to actually get that working through the UI. I'm pretty sure you couldn't even do it with an extension at this point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:52, 4 January 2024 (UTC)

:: I see - thanks for looking into it, and for the explanation! It would have been a nice option, but it's not a big deal.--[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 05:03, 4 January 2024 (UTC)

==Username Change (Zebendal)==
Hello, I'd like to change my username to: Analeah Oaksong --[[User:Zebendal|Zebendal]] ([[User talk:Zebendal|talk]]) 02:49, 6 January 2024 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:59, 11 January 2024 (UTC)

==Another vandalizing IP address-only account==
159.2.53.105 - several nonsense/"blanking" edits to Skyrim pages today that have yet to be reverted. {{unsigned|Mikeprichard|21:35, 10 January 2024 (UTC)}}
:172.251.169.141 - and another --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 02:24, 11 January 2024 (UTC)
::Looks like Jeancey got the first one and I warned and revision-deleted the second one a little earlier. Thanks! – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:28, 11 January 2024 (UTC)
86.29.213.176 [https://en.uesp.net/wiki/Special:Contributions/86.29.213.176] - thanks, all; here's yet another. IP accounts being vandals continues to be the rule, not the exception, with at least daily incidents over the past couple weeks that I've noticed. --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:Warned. As for IP accounts being primarily vandals, {{Pl|Special:RecentChanges|Recent Changes|3=hideliu=1}} doesn't back you up on that. As of 14:27 (UTC) today, there have been 144 edits by 10 different IP addresses and two edits by one IP which appear to have been vandalism. Even if, for some reason, you discount the one prolific IP editor, there's still nearly a 10:1 ratio of good IP edits to vandalism. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:13, 13 January 2024 (UTC)
::If that's the case (and I've certainly noticed the single extremely prolific IP editor making constructive edits), fair enough - I'm more just frustrated on your/the UESP's behalf that so much time must be spent on this incessant nonsense in order to maintain the integrity of the wiki. Some real nutjobs out there (other than me, I mean)! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 16:19, 13 January 2024 (UTC)
:::Hehehe. Unfortunately, that goes with the territory. Restricting edits to accounts only would almost certainly reduce the amount of vandalism, but some people are still willing to create accounts in order to vandalize, and in the meantime, we lose those edits by IP editors who don't feel their edit is worth creating an account for (which I'm guilty of myself on other wikis). – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:24, 13 January 2024 (UTC)

== Username Change Request (Thal-J) ==

Hi, just wanted to drop the dash from my name to be consistent with my Discord. So, "ThalJ". [[User:Thal-J|Thal-J]] ([[User talk:Thal-J|talk]]) 13:09, 24 January 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:05, 24 January 2024 (UTC)

== Patroller Access to MediaWiki space ==

While looking at how permissions work on the Starfield wiki, a few editors noticed that there were some changes in how access to the MediaWiki namespace was handled in later versions of MediaWiki. The reason for that is to limit even administrators' access to edit pages that could be used to really mess up the wiki or create security vulnerabilities. Respecting WMF's reasoning, I've separated out the ability to edit MediaWiki space from Patrollers and turned it into its own special group. This also allows employees like ThalJ to access it without needing any additional rights. At least for now, administrators' access remains unchanged, though some things will change automatically once we get to MW 1.32. For the sake of not overcomplicating this post, I'll save that discussion for whenever the time comes.

For now, this means that any patrollers who want to continue to be able to edit MediaWiki space will need to ask me or another admin for access to it. (Admins: you can do this by selecting "interface editor" in their [[Special:UserRights|groups]].) Since we've agreed in the past that the wiki benefits from patrollers having access to MediaWiki, this is free for the asking, but for those same security reasons I mentioned, it makes sense not to give access to patrollers who will never use it or who feel less comfortable in that space. Feel free to reply here or tag/DM me on Discord if you're a patroller and would like access to MediaWiki space again. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:11, 29 January 2024 (UTC)

:Thank you, RobinHood. It's good to have these permissions separate, just like how we have blocker rights separate (there are a lot of inactive patrollers). I can't think of any patrollers off-hand who would need it right now, so we'll have to wait for requests. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 04:16, 29 January 2024 (UTC)

== ESO Scions of Ithelia PTS Release (Update 41) ==

:* uespLog -- Might need one update in the sales data to not error.
:* File Uploads -- [https://esofiles.uesp.net/update-41pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=41pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=41pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- New Maps Added/Uploaded!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:07, 9 February 2024 (UTC)

== Vandalism blocked ==

Blocked [[Special:Contributions/Beaneater00|Beaneater00]] for [https://en.uesp.net/w/index.php?title=Morrowind:Gentleman_Jim_Stacey&curid=19417&action=history inserting nonsense] into page. User was [[User_talk:Beaneater00|warned]] and persisted in making disruptive edits. Reasonable time was allowed for the user to see the warning (two further edits were made after warning was issued). Block will require an extension. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 13:10, 18 February 2024 (UTC)
:Changed the block to indefinite. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 18 February 2024 (UTC)

== New Bureaucrat ==

Just a small announcement to say that [[User:RobinHood70|RobinHood70]] has been granted the [[UESPWiki:Bureaucrats|Bureaucrat]] role by Daveh. This is UESP's first new Bureaucrat since Nephele's departure. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 17:02, 18 February 2024 (UTC)

:Nice. Shoutout to Robin for bein such a chill guy [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 17:57, 18 February 2024 (UTC)

::Great choice. Much better to have Robin with these rights than Nephele, who has been inactive for years. It was only a few weeks ago I suggested having those rights taken off. I'm confident he'll use the rights well, in the few cases they apply. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 19:04, 18 February 2024 (UTC)

:::You already got my congratulations, but I'll reiterate that you're the perfect candidate for this. Happy to have a new bureaucrat on our team! --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 21:02, 18 February 2024 (UTC)

== Repeat vandal blocked ==

I have [[User_talk:159.2.53.105|re-blocked]] a previously-blocked user whose block has expired and who has returned and made [[Special:Contributions/159.2.53.105|further disruptive edits]] to the wiki. Block will need to be extended. Cheers! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:30, 25 February 2024 (UTC)

Edit: And both RobinHood and MolagBallet already got it. Thanks! — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 20:38, 25 February 2024 (UTC)

== Administrator etiquette ==

I was hoping a neutral admin, perhaps one that wasn’t involved in the discussion could take a look at some of the recent edits and edit summaries of Jeancey, particularly the ones found on [[Lore Talk:Skyrim|this page]]. The edit summaries that are passive aggressive are one matter, but some of the latest posts in the talk page itself are getting dangerously close to ad hominem if not already crossing that threshold. I would assume some of this stuff is pushing the boundaries of the etiquette guidelines for a standard user, nevermind an administrator who should serve as a standard setter for professionalism and proper etiquette. [[User:Dcking20|Dcking20]] ([[User talk:Dcking20|talk]]) 22:56, 25 February 2024 (UTC)
:While I agree that things started to go off the rails a little bit, it appears that they've recovered now, at least from what I can see. If you think I'm missing something egregious, feel free to DM me on Discord or send me an e-mail and point it out. I think the existence of this post in itself will serve as a reminder to everyone to try to keep their tempers under control, but I honestly don't see anything that needs any further action than that. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 02:14, 26 February 2024 (UTC)

== ESO Scions of Ithelia Release (Update 40) ==

:* uespLog -- Updated to 3.22 to fix a crash due to an API change.
:* File Uploads -- [https://esofiles.uesp.net/update-41/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- No updated maps.
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 20:21, 11 March 2024 (UTC)

== Talk page created in error ==

I accidentally made a talk page post on the wrong page. There was no prior content on the talk page. Not sure if it is standard policy to delete such pages or not but if so its [[Template talk:Heroes]]. My bad! [[User:Mindtrait0r|Mindtrait0r]] ([[User talk:Mindtrait0r|talk]]) 14:18, 26 March 2024 (UTC)

== Undo page move request for Daggerfall:On Oblivion ==

I'd appreciate someone taking a look at this minor issue I brought this up on the [[Daggerfall talk:On Oblivion|talk page]]. It looks like [[Daggerfall:On Oblivion]] was moved in error. [[User:Boustrophedon|Boustrophedon]] ([[User talk:Boustrophedon|talk]]) 10:49, 14 April 2024 (UTC)
:Moved, and replied to your talk page post there. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:38, 14 April 2024 (UTC)

== Interface Editor ==

Hi team - I was wondering if I could apply for Interface Editor permissions. That way I can keep on top of [[MediaWiki:Nsinfo-namespacelist|modspace pseudospaces]] that need to be created for the [[UESP:Modspace Project|Modspace Project]], as well as make any tweaks to [[MediaWiki:uespvector.css|Vector]] or the [[User:Enodoc#ESO Alliance Colours|ESO colours]] in CSS without having to pester admins to do it. Thanks! --[[User:Enodoc|Enodoc]] ([[User talk:Enodoc|talk]]) 21:42, 17 April 2024 (UTC)
:You are already an interface editor. [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 21:45, 17 April 2024 (UTC)

== ESO Gold Road PTS Release (Update 42) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-42pts/ Done!]
:* Champion Points -- [https://esolog.uesp.net/viewCps.php?version=42pts Done!]
:* Skills -- [https://esolog.uesp.net/viewSkills.php?version=42pts Done!]
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:19, 25 April 2024 (UTC)

== ESO Gold Road Release (Update 42) ==
:* uespLog -- Updated to 3.24 to some issues.
:* File Uploads -- [https://esofiles.uesp.net/update-43/ Done!]
:* Skills -- Done!
:* CP -- Done!
:* Skill Coefficients -- Done!
:* Icons/Images -- Done!
:* Items -- Done!
:* Maps -- Done!
:* Achievements -- Done!
:* Collectibles -- Done!
:* Book Data -- Done!
:* Recipe Data -- Done!
:* API -- Done!
:* Build Editor -- Mostly done....

-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:51, 12 June 2024 (UTC)

== UESP:ESO Alchemy Calculator, tab "Learn" ==

Hi,
I followed the first guide on the "Learn"-tab of page
: https://esolog.uesp.net/viewPotions.php
to discover all the traits.
: Quote: From level 1 Alchemy you can learn the traits of all reagents in '''31''' different combinations

But there are only '''30''' steps in this guide. And at the end there are traits left undiscovered for 2 reagents:
: Dragon's Bile: trait "Vulnerability"
: Fleshfly Larva: traits "Vulnerability" and "Vitality"
By combining the two reagents, all traits were discovered.

With the following 31st line the guide would be complete:
: <code><nowiki><li reagent1="Dragon's Bile" reagent2="Fleshfly Larva" reagent3="">Dragon's Bile + Fleshfly Larva</li></nowiki></code>

(Edit: The result was always the same with multiple characters.)

[[User:IceMan|IceMan]] ([[User talk:IceMan|talk]]) 01:41, 1 July 2024 (UTC)

:Thanks for the note, should be fixed in a minute. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:35, 17 July 2024 (UTC)

== ESO PTS (Update 43) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-43pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 18:33, 17 July 2024 (UTC)

== Interface Editor Request ==

To make minor updates to darkmode.css as/when needed '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 22:01, 23 July 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 22:22, 23 July 2024 (UTC)

== ESO PTS (Update 44) ==

:* uespLog -- Latest version seems to work fine.
:* File Uploads -- [https://esofiles.uesp.net/update-44pts/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Done!
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:34, 19 September 2024 (UTC)

== User Rights Request ==

Remove bot from [[User:CyrusBot|CyrusBot]] and administrator from me. —[[User:Dillonn241|Dillonn241]] ([[User talk:Dillonn241|talk]]) 17:23, 28 September 2024 (UTC)
:Done. If you change your mind in the near future, let me know and I'll restore them. Otherwise, you'll have to reapply for whatever roles through the usual routes. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:29, 28 September 2024 (UTC)

== Username Change Request (ViVYer) ==

Hello, I'd a username change from my current username to: Aanthirin. [[User:ViVYer|Aanthirin]] ([[User talk:ViVYer|talk]]) 22:13, 3 November 2024 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 23:57, 3 November 2024 (UTC)

== ESO (Update 44) ==

:* uespLog -- Updated to 3.25 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-44/ Done]
:* Champion Points -- Done!
:* Skills -- Done!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps --
:* Build Editor -- Mostly done!
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 16:08, 5 November 2024 (UTC)

== Username Change Request (CoolBlast3) ==
Hey! I'd like to change my current username to "Ithelia" if possible, thank you! [[User:CoolBlast3|CoolBlast3]] ([[User talk:CoolBlast3|talk]]) 21:24, 12 December 2024 (UTC)

: Done. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 21:26, 12 December 2024 (UTC)

== Possibility to view Recent Changes longer than 90 days ago? ==

I'm not sure whether this is the best page to ask this question, but is it possible to view Recent Changes that are older than 90 days ago? Or does one need to obtain certain privileges before one is able to do so? Simply asking since I've been volunteering since 2020 in watching all Oblivion and Shivering pages (both main and talk), where I do my best to review posts and questions of other users, but unfortunately I missed some days worth of changes made over 90 days ago. [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 00:20, 28 December 2024 (UTC)
:Sorry, no, it's not possible. Internally, Recent Changes are tracked in a separate table, and after 90 days, that info is deleted. The number of days can be changed by altering the server settings, but even if we did that, it's not retroactive. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:59, 28 December 2024 (UTC)
::And what if one is a Patroller? They don't receive auto-updates of changes made more than 90 days ago? [[User:C0rTeZ48|C0rTeZ48]] ([[User talk:C0rTeZ48|talk]]) 11:10, 4 January 2025 (UTC)
:::We don't receive updates, we just see Recent Changes and our Watchlist with (a lot of) {{FC|red|!}} marks for unpatrolled edits, and we have a [Mark as patrolled] button when we look at a diff. RC cuts off after 90 days for us, same as it does for you. If we look at anything older than that, either in RC or a diff, there's no exclamation mark and no button. For all intents, it's been patrolled at that point. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 15:20, 4 January 2025 (UTC)

== Edits to Tamriel Rebuilt:Ahhesha ==

There have recently been a number of low quality edits to [[Tamriel Rebuilt:Ahhesha]] from a couple new users and IPs. It doesn't look like it's vandalism necessarily; I think it might just be a new user still learning how to edit. Either way, I thought I'd let the admins know so you can give out the proper warnings and maybe protect the page if necessary. --[[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 15:35, 3 January 2025 (UTC)

== Requesting Username Change (Acynatic) ==
Hi, I'd like to request a username change from my current name to "Anariya". Thanks! [[User:Acynatic|Acynatic]] ([[User talk:Acynatic|talk]]) 12:03, 16 January 2025 (UTC)
:Done. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 16:01, 16 January 2025 (UTC)

== Request for admin opinion on a Skyrim talk:Redbelly Mine comment ==

To avoid escalating things into an edit war, I'd like to get get an admin's opinion on a message left on the [[Skyrim talk:Redbelly Mine]] page. The message, which read "Classic USSEP, changing random stuff that doesn't count as a bug. [[User:Histendarr|Histendarr]]", was deleted by [[User:Arthmoor]] for being a "Baseless attack against the USSEP". I reverted Arthmoor's edit given that the comment was left on a talk page. Arthmoor then reverted my edit, stating "This edit is blatant trolling and has no place here".

Personally, I don't see the message as either an attack or a troll. It isn't overly aggressive nor does it seem like it was intended to offend: it's just someone sharing their opinion on USKP/USSEP. I also question whether Arthmoor is the best person to decide whether this comment is acceptable or not, given his connection to USKP/USSEP. — [[User:Ohnoitsmangofett|Ohnoitsmangofett]] ([[User talk:Ohnoitsmangofett|talk]]) 20:37, 19 January 2025 (UTC)
:Since it's an older discussion, the comment qualifies as a necropost, plus it adds no useful information about the topic at hand, so it should remain off the page. You're right that Arthmoor would have a natural bias when it comes to USSEP, but that doesn't mean he's not allowed to remove comments like that, just that he has to be sure he can justify it when he's doing so. If there's any question about the neutrality of it, though, you should do exactly what you did and bring it up with uninvolved users for a second opinion. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:31, 20 January 2025 (UTC)

== Bluesky, Mastodon, Threads, and Youtube ==

We have social media presence on these sites and should link them on the sidebar.
*[https://bsky.app/profile/uesp.bsky.social Bluesky]
*[https://mstdn.ca/@uesp Mastodon]
*[https://www.threads.net/@unofficial_elder_scrolls_pages Threads]
*[https://www.youtube.com/@UnofficialElderScrollsPages Youtube]

We'll also want to update the Bluesky link to be a UESP link instead ultimately, but we'll need Daveh to do that. --[[User:Alpha Kenny Buddy|AKB]] [[User_talk:Alpha_Kenny_Buddy|Talk]] [[Special:Contributions/Alpha Kenny Buddy|Cont]] [[Special:Emailuser/Alpha Kenny Buddy|Mail]] 20:57, 1 February 2025 (UTC)

:I feel that a Threads link is redundant due to the Instagram link having such high overlap and already covering everything Threads does, though I feel the rest of the links to Bluesky, Mstdn, and YouTube should be added. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 21:11, 1 February 2025 (UTC)

::I'll try to get to work on this later this afternoon. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 18:53, 2 February 2025 (UTC)

:::And done. You may need to do a hard refresh. For now, I've included Threads as its own separate thing, but it's easy enough to take that out if we decide against it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 21:47, 2 February 2025 (UTC)

== Blocked Vandalism ==

Reporting that I blocked a vandal for repeatedly adding nonsense to pages, as required for a blocker. I think this one's pretty self-explanatory:

[[Special:Contributions/2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD|2A0A:EF40:E0D:C001:2007:76D0:F59B:B0DD]]

Block will need to be extended. Reported in the Discord channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 15:20, 1 March 2025 (UTC)
:Taken care of. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:49, 1 March 2025 (UTC)

== Dev has been upgraded to 1.31 ==

Just letting everyone know that dev has been upgraded to mw 1.31. This is a minor upgrade that shouldn't impact anything big, however anyone is welcome to run some tests on it. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 17:14, 6 March 2025 (UTC)
:It looks like the side search bar on dev isn't working quite right. I tried putting in <code>UESPWiki:MetaTemplate</code> and it just gave me an empty search results page. Putting the same thing in the search bar at the top of the page, however, works fine. I tried a few different times to make sure I hadn't just made a typo and I also tried switching between `UESPWiki` and `Project`, but it was the same thing every time.

:That might have something to do with UespCustomCode, I'm not sure. I haven't worked with that much. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 17:37, 9 March 2025 (UTC)

== ESO (Update 45) ==
:* uespLog -- Updated to 3.26 to fix a few things.
:* File Uploads -- [https://esofiles.uesp.net/update-45/ Done]
:* Champion Points -- Done!
:* Skills -- Done (Need to finish and check crafted skills)!
:* Skill Coefficients -- Done!
:* Items -- Done!
:* Icons -- Done!
:* Maps -- Nothing to Update
:* Build Editor -- Mostly done !
:* Achievements - Done!
:* Collectibles - Done!
:* Antiquities - Done!
:* API -- Done!
-- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 15:11, 12 March 2025 (UTC)

== Username Change (Mjolnir) ==

Hey, when I started this account I named it Mjolnir117. It was Mjolnir117 as recent as 2024 according to my email. If possible, could I get it changed back to Mjolnir117? I know for sure that I did not change it.
[[User:Mjolnir|Mjolnir]] ([[User talk:Mjolnir|talk]]) 04:40, 20 March 2025 (UTC)
:There's no record of any name changes to your account before now, so I'm not sure what happened there, but either way, you're Mjolnir117 now. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 12:58, 20 March 2025 (UTC)

== Upgrade to MediaWiki 1.31 ==

The upgrade of the main UESP site to 1.31 will be performed this Thursday (April 3rd) at 9am EST. There is no downtime expected but the search index will need to be rebuilt which will take a few days to complete.

Looks for updates here as the upgrade is performed and note any related issues below. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:09, 31 March 2025 (UTC)

:* Upgrade is complete. Search is being reindexed which will take a day or so to complete. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:41, 3 April 2025 (UTC)

:I've just gone through the release notes quickly and at a glance, I don't see any major changes in this version that would affect most users. You may notice slight formatting changes or new info/options on some special pages, but I think that's it. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 14:14, 3 April 2025 (UTC)

:A [https://www.reddit.com/r/ElderScrolls/comments/1jqt6pn/help_with_uesp/ user on Reddit] has reported issues with getting a 500 error when trying to log in - I've directed them to this page. ~ [[User:Alarra|Alarra]] ([[User_talk:Alarra|talk]] • [[Special:Contributions/Alarra|contribs]]) 21:58, 3 April 2025 (UTC)

== Username change request (Count Reman IV) ==

Good evening. Could I have my username changed to "Reman IV Nibenius" if possible? Thank you very much! - [[User:Count Reman IV|Count Reman IV]] [[User_talk:Count Reman IV|(talk)]] 23:30, 03 April 2025 (UTC)

: Done! —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 08:40, 4 April 2025 (UTC)

:: It appears that because your name was initially [[User:Count Reman IV & V of Cyrodiil]], and this is your second name change, some redirect targets will have to be updated to avoid [[wikipedia:Wikipedia:Double redirects|double redirects]]. I've fixed a few I spotted already, but just wanted to let you know as there might be more. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 08:50, 4 April 2025 (UTC)

== Request for Adminship: The Rim of the Sky ==

:''Our Request for Adminship (RfA) process is based on [[Wikipedia:Wikipedia:Requests_for_adminship|Wikipedia]]'s policy. An editor, when nominated, accepts and answers questions, which can be asked by any registered editor. Votes and comments can also be left by any registered editor. [[User:Daveh|Daveh]] makes the call, based upon the community's consensus, after about a week.''

{{Consensus|Support|2=Unanimous support. Moved to [[User:The Rim of the Sky/RfA]] upon completion of nomination. – [[User:RobinHood70|Robin Hood]]&ensp;[[User talk:RobinHood70|(talk)]] 04:06, 14 April 2025 (UTC)}}

== Username change request (Discranola) ==

If it's possible, could I have my username changed to MetalGearSurvive? Unless having my name literally just being the name of a video game would cause potential issues. I can try to think of something else if that's the case. Thank you! -- [[User:Discranola|Discranola]] ([[User talk:Discranola|talk]]) 11:58, 17 April 2025 (UTC)

:Hi there, I went ahead and renamed you. There are no rules against sharing a name with a video game, especially one that is non-TES related, so you should be fine. Let me know if there's anything else I can do! [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:56, 17 April 2025 (UTC)

== So I'm trying to fix a "Titles" header in [[Lore:Redguard Names#Skyrim]]... ==

...because it is simply formatted incorrectly.

And when trying to submit the edit, I get a warning: 
"Error: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: Chanel Filter".

No idea what this means. All I want to do is to make a very small alteration, replacing an asterisk with 8 equal signs (=).

Here is the code for the entire section I want to submit: 
<code><nowiki> ===Skyrim===</nowiki></code>
<code><nowiki>{{SurnameList|ns_base=Skyrim|Lu'ah Al-Skaven|[[Skyrim:Azzada Lylvieve|Lylvieve]]*|[[Skyrim:Baral Sendu|Sendu]]{{intnote|dark}}}}</nowiki></code>

<code><nowiki><nowiki>*</nowiki> ''This surname was taken from a Breton wife.'' </nowiki></code>

<code><nowiki>{{Note|dark||''This name fits the construction of a [[Lore:Dunmer Names|Dunmer name]] more so than Redguard. Whether this was a mistake or whether they were intended to have some back-story for this uncharacteristic name is impossible to guess, but the former is more likely given how inconsequential this character is in the game.}}</nowiki></code>

<code><nowiki>==== Titles ====</nowiki></code>
<code><nowiki>[[Skyrim:Majid the Mad|the Mad]]</nowiki></code>

I would appreciate if anyone'd help with this, thanks.
[[Special:Contributions/83.9.113.68|83.9.113.68]] 18:01, 21 April 2025 (UTC)

== How to rename wiki page ==
Is there a way to rename a ESO npc page? This is regarding with [[Online:Moon-Priest Haduras|Priest Haduras]], his current updated name should be Moon-Priest Haduras.
[[User:Raren|Raren]]
([[User talk:Raren|talk]]) 17:55, 27 April 2025 (ET)
:Yes, it's the "move page" option.[[User:Eve|—Eve]] ([[User talk:Eve|talk]]) 23:49, 27 April 2025 (UTC)
::I went ahead and moved the page. For Haduras in particular, it seems he was just called "Priest Haduras" at launch (Update 22) and then renamed to "Moon-Priest Haduras" in Update 23 for consistency with the rest of the priests. A bot auto-created the page "Moon-Priest Haduras" after the rename as a separate article, so it did not seem possible to move the page without first deleting the bot-generated article. I've gone ahead and moved "ON:Priest Haduras" overtop of "ON:Moon-Priest Haduras" so I think the issue should be fixed. [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 00:59, 28 April 2025 (UTC)

== Request for Cartographer ==

I was patrolling the maps talk page and would like to make a correction that was requested for a broken Morrowind map link. [[User:Thuraya Salaris|Thuraya Salaris]] ([[User talk:Thuraya Salaris|talk]]) 04:27, 30 May 2025 (UTC)

:Done --[[User:Ilaro|Ilaro]] ([[User talk:Ilaro|talk]]) 10:08, 30 May 2025 (UTC)

==Stepping back from the UESP - request for change to "inactive" patroller==
Hey, all - with our first child on the way and other personal interests taking up my increasingly limited free time, I'll need to step back from the wiki for at least the foreseeable future. I may come back for another burst of editing/reviewing at some point, but for now, I think I should be listed as an "inactive" patroller. I didn't think it was right to just make the change myself at [[UESPWiki:Patrollers#Current Patrollers]], but if I should, please let me know. Regardless, I wanted to take the opportunity to thank all the admins, fellow patrollers, and frequent contributors from whom I've learned a lot over the years, and to express appreciation for this community. Cheers! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 20:14, 5 June 2025 (UTC)
:{{User:Jeancey/Ah}} You could have done it yourself without issue but I've gonna ahead and made the change :) Congratulations on the kiddo! [[User:Jeancey|Jeancey]] ([[User talk:Jeancey|talk]]) 20:16, 5 June 2025 (UTC)
:{{Edit Conflict}} Hey Mike, just wanted to say thank you for all your help during your time here, you've been a great contributor for the wiki and we really appreciate all you've done. We've gone ahead and moved you to inactive, however you are more than welcome to set your own activity status (in fact it is usually preferred, we usually only adjust activity status of others if its found that they're not willing to do it themselves). If you do decide to come back one day when you have more free time, we'd all be more than happy to see you again. Congrats on the kid, and good luck with your future endeavors! :) [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 20:19, 5 June 2025 (UTC)
::Just to chip in and add to what's been said above... Mike, it's been an absolute pleasure to edit together and thank you so much for all your contributions :) Best of luck for your exciting new chapter! --[[User:SerCenKing|SerCenKing]] ([[User talk:SerCenKing|talk]]) 21:00, 5 June 2025 (UTC)
:::Thanks so much, everyone!! The feeling's mutual. Happy editing, and take care! --[[User:Mikeprichard|Mikeprichard]] ([[User talk:Mikeprichard|talk]]) 12:02, 6 June 2025 (UTC)

== Username Change Request (Fueranel) ==

Hello! Returning user from some years back, and I was hoping to have my username updated to one of which I go by on all platforms now. It's in fact one I used previously on this account, as going to it redirects to the name this account currently has. If I could have my username reverted to 'Kaebus', that would be teriffic! Thank you. --[[User:Fueranel|Fueranel]] ([[User talk:Fueranel|talk]]) 06:26, 13 June 2025 (UTC)

:Done! Since your original username already existed as a redirect I deleted it and manually moved your userpage to the original name as well, so it should all be taken care of now [[User:The Rim of the Sky|The Rim of the Sky]] ([[User talk:The Rim of the Sky|talk]]) 07:22, 13 June 2025 (UTC)

::Wonderful! All looks perfect. Thank you for your assistance, happy to be back! --[[User:Kaebus|Kaebus]] ([[User talk:Kaebus|talk]]) 07:24, 13 June 2025 (UTC)

== Request for Page Mover ==

I've been fixing a typo and, in the process, I moved four pages, updated other pages to link to the correct pages instead of the resulting redirects, then proposed the now-orphaned redirects for deletion [[Special:Diff/3383636|1]] [[Special:Diff/3383637|2]] [[Special:Diff/3383638|3]] [[Special:Diff/3383639|4]]. Only several minutes later did I remember the recently rolled-out role of Page Mover.

Although I do not foresee myself frequently making similar page moves, I believe it will be a useful tool for when I do. It would also be convenient for administrators if we could skip the whole speedy deletion process for cleanup. Rest assured, I am always diligent in ensuring my moves will not result in redlinks in other pages.

Thus, I would like to request for the Page Mover role, if I qualify. — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 19:12, 26 June 2025 (UTC)

: Page Mover requests belong on [[UESPWiki:Role Requests]] but I've gone ahead and granted this. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 10:00, 29 July 2025 (UTC)

:: Noted, thanks! — [[User:Salamangkero|Salamangkero]] ([[User_talk:Salamangkero|talk]]) 18:35, 29 July 2025 (UTC)

== UESP Upgraded to 1.32 ==

Uesp has been upgraded to 1.32. Please let me know either here or on the discord if you encounter any issues. [[User:Eckserah|Eckserah]] ([[User talk:Eckserah|talk]]) 19:30, 16 September 2025 (UTC)

== Blocked User ==

Blocked a user for six hours for [https://en.uesp.net/w/index.php?title=User:Tarponpet&curid=354691&action=history repeatedly making inappropriate edits] to another user's user page, despite being [[User talk:Ronnoc77|asked/warned multiple times]] to stop by multiple editors: [[Special:Contributions/Ronnoc77|Ronnoc77]]. The last edit included intimidating behaviour toward the other editor. An admin can review the edits and decide if the block should be extended. This was noted/discussed at length in the Discord's Patrolling channel as well. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 09:27, 20 January 2026 (UTC)

: Extended. —⁠[[User:Legoless|Legoless]] ([[User talk:Legoless|talk]]) 09:41, 20 January 2026 (UTC)
::Please also see [https://en.uesp.net/wiki/Special:Contributions/Ronnoc7777 new user], blocked in association with previous. '''[[User:Kiz|Kiz]]''' [[Special:EmailUser/Kiz|(email]] - [[User_Talk:Kiz|talk)]] 10:24, 20 January 2026 (UTC)

== Block Appeal ==

User [[Special:Contributions/Ronnoc77|Ronnoc 77]] has [[User talk:Ronnoc77#Appeal|appealed their current block]]. Posting notice here per block appeal procedure. — [[User:Wolfborn|Wolfborn]][[User_Talk:Wolfborn|(Howl)]] 11:03, 22 January 2026 (UTC)

== Server Upgrade ==

We are upgrading all servers in the coming weeks and status updates will be posted below. We don't expect any service interruptions during this upgrade. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:21, 15 April 2026 (UTC)

:* '''Files1 - 16 April 2026 10AM EST''' -- We are switching over the files/images server this morning. No service interruption is expected. There is a small chance that if you upload an image before the your DNS entries are updated the new image will not display. This is temporary and the image will display correctly once your DNS points to the new server. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 13:52, 16 April 2026 (UTC)
:* '''Search1 -- 17 April 2026 10AM EST''' -- We are planning on switching over search1 (ElasticSearch indexes) to the new server this morning. No service interruption is expected. We are pausing all wiki jobs during the transition to prevent any lost indexing updates. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:00, 17 April 2026 (UTC)
:* '''Db2 -- 19 April 2026 11AM EST''' -- We are switching over Db2 (database replication, primary read) server this morning. No service interruption is expected. -- [[User:Daveh|Daveh]] [[User_talk:Daveh|(talk)]] 14:45, 19 April 2026 (UTC)

=== Problem Reports ===
Post any issues that might be related to the server upgrade below.