khangtran
/
dark_web_forums
0
0
Fork 0
Code Issues 0 Pull Requests 0 Projects 0 Releases 0 Wiki Activity
Browse Source

fixed merge conflict issues

main
Khoi 1 year ago
parent
101e17e21e 33a5379c3e
commit
b404ffa59c
175 changed files with 53458 additions and 340 deletions
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +3
    -3
      .gitignore
  2. BIN
      Forums/AbyssForum/__pycache__/crawler_selenium.cpython-310.pyc
  3. BIN
      Forums/AbyssForum/__pycache__/crawler_selenium.cpython-311.pyc
  4. BIN
      Forums/AbyssForum/__pycache__/parser.cpython-310.pyc
  5. BIN
      Forums/AbyssForum/__pycache__/parser.cpython-311.pyc
  6. +462
    -0
      Forums/AbyssForum/geckodriver.log
  7. BIN
      Forums/Altenens/__pycache__/crawler_selenium.cpython-310.pyc
  8. BIN
      Forums/Altenens/__pycache__/crawler_selenium.cpython-311.pyc
  9. BIN
      Forums/Altenens/__pycache__/parser.cpython-310.pyc
  10. BIN
      Forums/Altenens/__pycache__/parser.cpython-311.pyc
  11. +13
    -13
      Forums/Altenens/crawler_selenium.py
  12. +462
    -0
      Forums/Altenens/geckodriver.log
  13. +67
    -192
      Forums/Altenens/parser.py
  14. BIN
      Forums/BestCardingWorld/__pycache__/__init__.cpython-311.pyc
  15. BIN
      Forums/BestCardingWorld/__pycache__/crawler_selenium.cpython-310.pyc
  16. BIN
      Forums/BestCardingWorld/__pycache__/crawler_selenium.cpython-311.pyc
  17. BIN
      Forums/BestCardingWorld/__pycache__/parser.cpython-310.pyc
  18. BIN
      Forums/BestCardingWorld/__pycache__/parser.cpython-311.pyc
  19. +462
    -0
      Forums/BestCardingWorld/geckodriver.log
  20. +331
    -0
      Forums/Cardingleaks/crawler_selenium.py
  21. +247
    -0
      Forums/Cardingleaks/parser.py
  22. BIN
      Forums/Classifier/__pycache__/__init__.cpython-310.pyc
  23. BIN
      Forums/Classifier/__pycache__/__init__.cpython-311.pyc
  24. BIN
      Forums/Classifier/__pycache__/classify_product.cpython-310.pyc
  25. BIN
      Forums/Classifier/__pycache__/classify_product.cpython-311.pyc
  26. BIN
      Forums/Classifier/__pycache__/transformer.cpython-310.pyc
  27. BIN
      Forums/Classifier/__pycache__/transformer.cpython-311.pyc
  28. BIN
      Forums/CryptBB/__pycache__/__init__.cpython-311.pyc
  29. BIN
      Forums/CryptBB/__pycache__/crawler_selenium.cpython-310.pyc
  30. BIN
      Forums/CryptBB/__pycache__/crawler_selenium.cpython-311.pyc
  31. BIN
      Forums/CryptBB/__pycache__/parser.cpython-310.pyc
  32. BIN
      Forums/CryptBB/__pycache__/parser.cpython-311.pyc
  33. BIN
      Forums/CryptBB/captcha.png
  34. +462
    -0
      Forums/CryptBB/geckodriver.log
  35. +1
    -1
      Forums/CryptBB/parser.py
  36. BIN
      Forums/DB_Connection/__pycache__/__init__.cpython-310.pyc
  37. BIN
      Forums/DB_Connection/__pycache__/__init__.cpython-311.pyc
  38. BIN
      Forums/DB_Connection/__pycache__/db_connection.cpython-310.pyc
  39. BIN
      Forums/DB_Connection/__pycache__/db_connection.cpython-311.pyc
  40. BIN
      Forums/HiddenAnswers/__pycache__/crawler_selenium.cpython-310.pyc
  41. BIN
      Forums/HiddenAnswers/__pycache__/crawler_selenium.cpython-311.pyc
  42. BIN
      Forums/HiddenAnswers/__pycache__/parser.cpython-310.pyc
  43. BIN
      Forums/HiddenAnswers/__pycache__/parser.cpython-311.pyc
  44. +462
    -0
      Forums/HiddenAnswers/geckodriver.log
  45. BIN
      Forums/Initialization/__pycache__/__init__.cpython-310.pyc
  46. BIN
      Forums/Initialization/__pycache__/__init__.cpython-311.pyc
  47. BIN
      Forums/Initialization/__pycache__/forums_mining.cpython-310.pyc
  48. BIN
      Forums/Initialization/__pycache__/forums_mining.cpython-311.pyc
  49. BIN
      Forums/Initialization/__pycache__/prepare_parser.cpython-310.pyc
  50. BIN
      Forums/Initialization/__pycache__/prepare_parser.cpython-311.pyc
  51. +9
    -6
      Forums/Initialization/forums_mining.py
  52. +10953
    -0
      Forums/Initialization/geckodriver.log
  53. +330
    -0
      Forums/Libre/crawler_selenium.py
  54. +247
    -0
      Forums/Libre/parser.py
  55. BIN
      Forums/OnniForums/__pycache__/crawler_selenium.cpython-310.pyc
  56. BIN
      Forums/OnniForums/__pycache__/crawler_selenium.cpython-311.pyc
  57. BIN
      Forums/OnniForums/__pycache__/parser.cpython-310.pyc
  58. BIN
      Forums/OnniForums/__pycache__/parser.cpython-311.pyc
  59. BIN
      Forums/OnniForums/__pycache__/parser_script.cpython-311.pyc
  60. +16
    -16
      Forums/OnniForums/crawler_selenium.py
  61. +462
    -0
      Forums/OnniForums/geckodriver.log
  62. +320
    -0
      Forums/Procrax/crawler_selenium.py
  63. +264
    -0
      Forums/Procrax/parser.py
  64. BIN
      Forums/Utilities/__pycache__/__init__.cpython-310.pyc
  65. BIN
      Forums/Utilities/__pycache__/__init__.cpython-311.pyc
  66. BIN
      Forums/Utilities/__pycache__/utilities.cpython-310.pyc
  67. BIN
      Forums/Utilities/__pycache__/utilities.cpython-311.pyc
  68. BIN
      Forums/__pycache__/__init__.cpython-310.pyc
  69. BIN
      MarketPlaces/AnonymousMarketplace/__pycache__/crawler_selenium.cpython-310.pyc
  70. BIN
      MarketPlaces/AnonymousMarketplace/__pycache__/crawler_selenium.cpython-311.pyc
  71. BIN
      MarketPlaces/AnonymousMarketplace/__pycache__/parser.cpython-310.pyc
  72. BIN
      MarketPlaces/AnonymousMarketplace/__pycache__/parser.cpython-311.pyc
  73. +1483
    -0
      MarketPlaces/AnonymousMarketplace/geckodriver.log
  74. BIN
      MarketPlaces/Apocalypse/__pycache__/crawler_selenium.cpython-310.pyc
  75. BIN
      MarketPlaces/Apocalypse/__pycache__/crawler_selenium.cpython-311.pyc
  76. BIN
      MarketPlaces/Apocalypse/__pycache__/parser.cpython-310.pyc
  77. BIN
      MarketPlaces/Apocalypse/__pycache__/parser.cpython-311.pyc
  78. +1483
    -0
      MarketPlaces/Apocalypse/geckodriver.log
  79. BIN
      MarketPlaces/BlackPyramid/__pycache__/crawler_selenium.cpython-310.pyc
  80. BIN
      MarketPlaces/BlackPyramid/__pycache__/parser.cpython-310.pyc
  81. +1483
    -0
      MarketPlaces/BlackPyramid/geckodriver.log
  82. BIN
      MarketPlaces/CityMarket/__pycache__/crawler_selenium.cpython-310.pyc
  83. BIN
      MarketPlaces/CityMarket/__pycache__/crawler_selenium.cpython-311.pyc
  84. BIN
      MarketPlaces/CityMarket/__pycache__/parser.cpython-310.pyc
  85. BIN
      MarketPlaces/CityMarket/__pycache__/parser.cpython-311.pyc
  86. +1483
    -0
      MarketPlaces/CityMarket/geckodriver.log
  87. BIN
      MarketPlaces/Classifier/__pycache__/__init__.cpython-310.pyc
  88. BIN
      MarketPlaces/Classifier/__pycache__/__init__.cpython-311.pyc
  89. BIN
      MarketPlaces/Classifier/__pycache__/classify_product.cpython-310.pyc
  90. BIN
      MarketPlaces/Classifier/__pycache__/classify_product.cpython-311.pyc
  91. BIN
      MarketPlaces/Classifier/__pycache__/transformer.cpython-310.pyc
  92. BIN
      MarketPlaces/Classifier/__pycache__/transformer.cpython-311.pyc
  93. +1483
    -0
      MarketPlaces/CypherMarketplace/geckodriver.log
  94. BIN
      MarketPlaces/DB_Connection/__pycache__/__init__.cpython-310.pyc
  95. BIN
      MarketPlaces/DB_Connection/__pycache__/__init__.cpython-311.pyc
  96. BIN
      MarketPlaces/DB_Connection/__pycache__/db_connection.cpython-310.pyc
  97. BIN
      MarketPlaces/DB_Connection/__pycache__/db_connection.cpython-311.pyc
  98. BIN
      MarketPlaces/DarkFox/__pycache__/crawler_selenium.cpython-310.pyc
  99. BIN
      MarketPlaces/DarkFox/__pycache__/crawler_selenium.cpython-311.pyc
  100. BIN
      MarketPlaces/DarkFox/__pycache__/parser.cpython-310.pyc

+ 3
- 3
.gitignore View File

@ -1,8 +1,8 @@
__pycache__
# Default ignored files
/shelf/ /shelf/
workspace.xml
.idea/workspace.xml
selenium/geckodriver.exe selenium/geckodriver.exe
setup.ini setup.ini
*.html *.html
*.log *.log
*.png
*.png

BIN
Forums/AbyssForum/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
Forums/AbyssForum/__pycache__/crawler_selenium.cpython-311.pyc View File


BIN
Forums/AbyssForum/__pycache__/parser.cpython-310.pyc View File


BIN
Forums/AbyssForum/__pycache__/parser.cpython-311.pyc View File


+ 462
- 0
Forums/AbyssForum/geckodriver.log View File

@ -0,0 +1,462 @@
1655762162153 geckodriver INFO Listening on 127.0.0.1:50333
1655762166434 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50334" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqUeAAN"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762166904 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50334/devtools/browser/fb880205-35da-44a4-83b8-a861ce7125f1
1655762168590 Marionette INFO Listening on port 50341
1655762169104 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655762207624 Marionette INFO Stopped listening on port 50341
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655762207923 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655762990320 geckodriver INFO Listening on 127.0.0.1:50884
1655762994595 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50885" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileurjxEe"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762995244 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50885/devtools/browser/36503f58-7ab7-4188-81b0-fa1b81f32c0a
1655762996997 Marionette INFO Listening on port 50890
JavaScript error: resource://gre/modules/ExtensionContent.jsm, line 575: TypeError: PrecompiledScript.executeInGlobal: Argument 1 is not an object.
1655762997277 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763083091 Marionette INFO Stopped listening on port 50890
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763083216 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763089557 geckodriver INFO Listening on 127.0.0.1:50923
1655763093759 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50924" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile5mY1qq"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763094208 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50924/devtools/browser/68ce2df9-521f-4258-94f7-c2e2b199701a
1655763095918 Marionette INFO Listening on port 50929
1655763096408 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763143935 Marionette INFO Stopped listening on port 50929
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763144495 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763166975 geckodriver INFO Listening on 127.0.0.1:50947
1655763171175 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50948" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilenAGZM5"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763171719 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50948/devtools/browser/0b9966bd-0cec-45cc-9d11-02a2580233f0
1655763173307 Marionette INFO Listening on port 50953
1655763173368 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763208922 Marionette INFO Stopped listening on port 50953
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763209261 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763387409 geckodriver INFO Listening on 127.0.0.1:49752
1655763391806 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "49753" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilevAHJSJ"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763392827 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:49753/devtools/browser/7fe49a16-940f-4ee8-9a2f-48739a92db78
1655763394700 Marionette INFO Listening on port 49759
1655763395103 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763430848 Marionette INFO Stopped listening on port 49759
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763431145 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655765589286 geckodriver INFO Listening on 127.0.0.1:50102
1655765621677 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50103" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ecUpb"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655765622149 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50103/devtools/browser/e5a916fc-987c-43d4-9c12-9c9b88cca242
1655765623776 Marionette INFO Listening on port 50109
1655765623892 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766125168 Marionette INFO Stopped listening on port 50109
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655766125303 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655766505099 geckodriver INFO Listening on 127.0.0.1:50156
1655766509351 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50157" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileVqvmzh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655766509876 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50157/devtools/browser/13084c14-64d3-48a5-8b99-4c514e961d4f
1655766511328 Marionette INFO Listening on port 50162
1655766511395 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766552298 Marionette INFO Stopped listening on port 50162
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655766552578 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767657126 geckodriver INFO Listening on 127.0.0.1:50257
1655767661375 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50258" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileB9Dzeh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767661843 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50258/devtools/browser/867cd318-6430-4b3f-9b63-8c6bdf17636a
1655767663330 Marionette INFO Listening on port 50263
1655767663431 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767668263 Marionette INFO Stopped listening on port 50263
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655767668591 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767689480 geckodriver INFO Listening on 127.0.0.1:50276
1655767693635 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50277" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ZOQe7"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767694215 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50277/devtools/browser/a70a3f5b-19af-4a68-99c8-c46086ba9599
1655767695725 Marionette INFO Listening on port 50282
1655767695830 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767701272 Marionette INFO Stopped listening on port 50282
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767701594 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767791360 geckodriver INFO Listening on 127.0.0.1:50296
1655767795575 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50297" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile4GEqBr"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767796001 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50297/devtools/browser/e2d4e57a-16ce-429d-9804-e9e391199ddf
1655767797534 Marionette INFO Listening on port 50302
1655767797661 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767802833 Marionette INFO Stopped listening on port 50302
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767803151 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767902908 geckodriver INFO Listening on 127.0.0.1:50320
1655767907009 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50321" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqlJ8ZA"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767907484 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50321/devtools/browser/49e77669-01dd-447e-a804-36d42b3400cf
1655767908782 Marionette INFO Listening on port 50326
1655767909069 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767916218 Marionette INFO Stopped listening on port 50326
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767916523 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767958348 geckodriver INFO Listening on 127.0.0.1:50340
1655767962557 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50341" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilekpCdfs"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767963032 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50341/devtools/browser/b4d1728c-d26f-4537-adc2-5c954b171b13
1655767964591 Marionette INFO Listening on port 50346
1655767964633 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767973210 Marionette INFO Stopped listening on port 50346
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767973501 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767998084 geckodriver INFO Listening on 127.0.0.1:50358
1655768002314 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50359" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileujYNBj"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768002757 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50359/devtools/browser/04112c45-204d-43f1-9615-34782fd06632
1655768004165 Marionette INFO Listening on port 50364
1655768004375 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768009787 Marionette INFO Stopped listening on port 50364
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655768010100 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768083966 geckodriver INFO Listening on 127.0.0.1:50386
1655768088234 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50387" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile39Pl1f"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768088664 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50387/devtools/browser/3d6cda42-cb60-4c67-8d76-1d186a51887f
1655768090263 Marionette INFO Listening on port 50392
1655768090299 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768091655 Marionette INFO Stopped listening on port 50392
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
1655768092659 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768101201 geckodriver INFO Listening on 127.0.0.1:50404
1655768105373 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50405" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile2YRGSz"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768105853 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50405/devtools/browser/9c4a4d71-25ff-4980-a54d-8545d6200790
1655768107431 Marionette INFO Listening on port 50410
1655768107530 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768119155 Marionette INFO Stopped listening on port 50410
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655768119438 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768229472 geckodriver INFO Listening on 127.0.0.1:50431
1655768233458 webdriver::server WARN Rejected request with missing Host header

BIN
Forums/Altenens/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
Forums/Altenens/__pycache__/crawler_selenium.cpython-311.pyc View File


BIN
Forums/Altenens/__pycache__/parser.cpython-310.pyc View File


BIN
Forums/Altenens/__pycache__/parser.cpython-311.pyc View File


+ 13
- 13
Forums/Altenens/crawler_selenium.py View File

@ -1,7 +1,8 @@
__author__ = 'Helium' __author__ = 'Helium'
''' '''
Altenens Forum Crawler (Selenium)
Altenens Forum Crawler (Selenium);
Untested due to CAPTCHAS and blocking the network
''' '''
from selenium import webdriver from selenium import webdriver
@ -36,7 +37,7 @@ def startCrawling():
if driver != 'down': if driver != 'down':
try: try:
# login(driver)
login(driver)
crawlForum(driver) crawlForum(driver)
except Exception as e: except Exception as e:
print(driver.current_url, e) print(driver.current_url, e)
@ -61,23 +62,23 @@ def opentor():
# Login using premade account credentials and do login captcha manually # Login using premade account credentials and do login captcha manually
def login(driver): def login(driver):
#click login button #click login button
login = driver.find_element(by=By.XPATH, value='//*[@id="top"]/div[1]/div/div/div/div[1]/a[1]')
login.click()
login_link = driver.find_element(by=By.XPATH, value='/html/body/div[1]/div[1]/div/div/div/div[1]/a[1]').get_attribute('href')
driver.get(login_link) # open tab with url
#entering username and password into input boxes #entering username and password into input boxes
usernameBox = driver.find_element(by=By.XPATH, value='//*[@id="_xfUid-1-1688066635"]')
usernameBox = driver.find_element(by=By.XPATH, value='/html/body/div[1]/div[4]/div/div/div[3]/div/div/div/form/div[1]/div/dl[1]/dd/input')
#Username here #Username here
usernameBox.send_keys('mylittlepony45')#sends string to the username box usernameBox.send_keys('mylittlepony45')#sends string to the username box
passwordBox = driver.find_element(by=By.XPATH, value='//*[@id="_xfUid-2-1688066635"]')
passwordBox = driver.find_element(by=By.XPATH, value='/html/body/div[1]/div[4]/div/div/div[3]/div/div/div/form/div[1]/div/dl[2]/dd/div/div/input')
#Password here #Password here
passwordBox.send_keys('johnnyTest@18')# sends string to passwordBox passwordBox.send_keys('johnnyTest@18')# sends string to passwordBox
input("Press ENTER when CAPTCHA is completed\n")
input("Press ENTER when you complete the CAPTCHA and press login\n")
# wait for listing page show up (This Xpath may need to change based on different seed url) # wait for listing page show up (This Xpath may need to change based on different seed url)
# wait for 50 sec until id = tab_content is found, then cont # wait for 50 sec until id = tab_content is found, then cont
WebDriverWait(driver, 50).until(EC.visibility_of_element_located(
(By.XPATH, '///html/body/div[1]/div[4]/div/div/div[3]/div/div/div[4]/div/div/div[1]/div/div[1]/div[2]/ol/li[1]')))
# WebDriverWait(driver, 50).until(EC.visibility_of_element_located(
# (By.XPATH, '/html/body/div[1]/div[4]/div/div/div[3]/div/div/div[4]/div/div/div[1]/div/div[1]')))
# Returns the name of the website # Returns the name of the website
@ -241,9 +242,9 @@ def crawlForum(driver):
savePage(driver.page_source, item) savePage(driver.page_source, item)
# if there is a next page then go and save.... # if there is a next page then go and save....
# next page in the topic?
# specific
try: try:
item = driver.find_element(By.XPATH, '/html/body/div[1]/div[4]/div/div/div[3]/div/div/div[1]/div[1]/div[1]/nav/div[1]/a').get_attribute('href')
item = driver.find_element(By.LINK_TEXT, value='Next').get_attribute('href')
if item == "": if item == "":
raise NoSuchElementException raise NoSuchElementException
@ -265,8 +266,7 @@ def crawlForum(driver):
break break
try:# change depending on web page, #next page try:# change depending on web page, #next page
link = driver.find_element(by=By.XPATH, value = '/html/body/div[1]/div[4]/div/div/div[3]/div/div/div/div[1]/div[1]/nav/div[1]/a').get_attribute('href')
link = driver.find_element(by=By.LINK_TEXT, value='Next').get_attribute('href')
if link == "": if link == "":
raise NoSuchElementException raise NoSuchElementException
try: try:


+ 462
- 0
Forums/Altenens/geckodriver.log View File

@ -0,0 +1,462 @@
1655762162153 geckodriver INFO Listening on 127.0.0.1:50333
1655762166434 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50334" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqUeAAN"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762166904 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50334/devtools/browser/fb880205-35da-44a4-83b8-a861ce7125f1
1655762168590 Marionette INFO Listening on port 50341
1655762169104 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655762207624 Marionette INFO Stopped listening on port 50341
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655762207923 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655762990320 geckodriver INFO Listening on 127.0.0.1:50884
1655762994595 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50885" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileurjxEe"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762995244 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50885/devtools/browser/36503f58-7ab7-4188-81b0-fa1b81f32c0a
1655762996997 Marionette INFO Listening on port 50890
JavaScript error: resource://gre/modules/ExtensionContent.jsm, line 575: TypeError: PrecompiledScript.executeInGlobal: Argument 1 is not an object.
1655762997277 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763083091 Marionette INFO Stopped listening on port 50890
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763083216 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763089557 geckodriver INFO Listening on 127.0.0.1:50923
1655763093759 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50924" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile5mY1qq"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763094208 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50924/devtools/browser/68ce2df9-521f-4258-94f7-c2e2b199701a
1655763095918 Marionette INFO Listening on port 50929
1655763096408 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763143935 Marionette INFO Stopped listening on port 50929
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763144495 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763166975 geckodriver INFO Listening on 127.0.0.1:50947
1655763171175 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50948" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilenAGZM5"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763171719 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50948/devtools/browser/0b9966bd-0cec-45cc-9d11-02a2580233f0
1655763173307 Marionette INFO Listening on port 50953
1655763173368 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763208922 Marionette INFO Stopped listening on port 50953
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763209261 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763387409 geckodriver INFO Listening on 127.0.0.1:49752
1655763391806 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "49753" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilevAHJSJ"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763392827 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:49753/devtools/browser/7fe49a16-940f-4ee8-9a2f-48739a92db78
1655763394700 Marionette INFO Listening on port 49759
1655763395103 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763430848 Marionette INFO Stopped listening on port 49759
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763431145 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655765589286 geckodriver INFO Listening on 127.0.0.1:50102
1655765621677 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50103" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ecUpb"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655765622149 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50103/devtools/browser/e5a916fc-987c-43d4-9c12-9c9b88cca242
1655765623776 Marionette INFO Listening on port 50109
1655765623892 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766125168 Marionette INFO Stopped listening on port 50109
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655766125303 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655766505099 geckodriver INFO Listening on 127.0.0.1:50156
1655766509351 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50157" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileVqvmzh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655766509876 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50157/devtools/browser/13084c14-64d3-48a5-8b99-4c514e961d4f
1655766511328 Marionette INFO Listening on port 50162
1655766511395 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766552298 Marionette INFO Stopped listening on port 50162
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655766552578 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767657126 geckodriver INFO Listening on 127.0.0.1:50257
1655767661375 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50258" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileB9Dzeh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767661843 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50258/devtools/browser/867cd318-6430-4b3f-9b63-8c6bdf17636a
1655767663330 Marionette INFO Listening on port 50263
1655767663431 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767668263 Marionette INFO Stopped listening on port 50263
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655767668591 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767689480 geckodriver INFO Listening on 127.0.0.1:50276
1655767693635 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50277" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ZOQe7"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767694215 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50277/devtools/browser/a70a3f5b-19af-4a68-99c8-c46086ba9599
1655767695725 Marionette INFO Listening on port 50282
1655767695830 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767701272 Marionette INFO Stopped listening on port 50282
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767701594 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767791360 geckodriver INFO Listening on 127.0.0.1:50296
1655767795575 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50297" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile4GEqBr"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767796001 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50297/devtools/browser/e2d4e57a-16ce-429d-9804-e9e391199ddf
1655767797534 Marionette INFO Listening on port 50302
1655767797661 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767802833 Marionette INFO Stopped listening on port 50302
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767803151 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767902908 geckodriver INFO Listening on 127.0.0.1:50320
1655767907009 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50321" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqlJ8ZA"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767907484 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50321/devtools/browser/49e77669-01dd-447e-a804-36d42b3400cf
1655767908782 Marionette INFO Listening on port 50326
1655767909069 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767916218 Marionette INFO Stopped listening on port 50326
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767916523 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767958348 geckodriver INFO Listening on 127.0.0.1:50340
1655767962557 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50341" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilekpCdfs"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767963032 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50341/devtools/browser/b4d1728c-d26f-4537-adc2-5c954b171b13
1655767964591 Marionette INFO Listening on port 50346
1655767964633 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767973210 Marionette INFO Stopped listening on port 50346
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767973501 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767998084 geckodriver INFO Listening on 127.0.0.1:50358
1655768002314 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50359" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileujYNBj"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768002757 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50359/devtools/browser/04112c45-204d-43f1-9615-34782fd06632
1655768004165 Marionette INFO Listening on port 50364
1655768004375 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768009787 Marionette INFO Stopped listening on port 50364
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655768010100 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768083966 geckodriver INFO Listening on 127.0.0.1:50386
1655768088234 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50387" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile39Pl1f"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768088664 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50387/devtools/browser/3d6cda42-cb60-4c67-8d76-1d186a51887f
1655768090263 Marionette INFO Listening on port 50392
1655768090299 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768091655 Marionette INFO Stopped listening on port 50392
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
1655768092659 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768101201 geckodriver INFO Listening on 127.0.0.1:50404
1655768105373 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50405" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile2YRGSz"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768105853 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50405/devtools/browser/9c4a4d71-25ff-4980-a54d-8545d6200790
1655768107431 Marionette INFO Listening on port 50410
1655768107530 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768119155 Marionette INFO Stopped listening on port 50410
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655768119438 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768229472 geckodriver INFO Listening on 127.0.0.1:50431
1655768233458 webdriver::server WARN Rejected request with missing Host header

+ 67
- 192
Forums/Altenens/parser.py View File

@ -1,4 +1,4 @@
__author__ = 'Helium'
__author__ = 'DarkWeb'
# Here, we are importing the auxiliary functions to clean or convert data # Here, we are importing the auxiliary functions to clean or convert data
from Forums.Utilities.utilities import * from Forums.Utilities.utilities import *
@ -11,162 +11,56 @@ from bs4 import BeautifulSoup
# This is the method to parse the Description Pages (one page to each topic in the Listing Pages) # This is the method to parse the Description Pages (one page to each topic in the Listing Pages)
def cryptBB_description_parser(soup):
# Fields to be parsed
def altenens_description_parser(soup):
topic = "-1" # topic name
user = [] # all users of each post
addDate = [] # all dated of each post
feedback = [] # all feedbacks of each vendor (this was found in just one Forum and with a number format)
status = [] # all user's authority in each post such as (adm, member, dangerous)
reputation = [] # all user's karma in each post (usually found as a number)
sign = [] # all user's signature in each post (usually a standard message after the content of the post)
post = [] # all messages of each post
interest = [] # all user's interest in each post
topic = "-1" # 0 *topic name
user = [] # 1 *all users of each post
status = [] # 2 all user's authority in each post such as (adm, member, dangerous)
reputation = [] # 3 all user's karma in each post (usually found as a number)
interest = [] # 4 all user's interest in each post
sign = [] # 5 all user's signature in each post (usually a standard message after the content of the post)
post = [] # 6 all messages of each post
feedback = [] # 7 all feedbacks of each vendor (this was found in just one Forum and with a number format)
addDate = [] # 8 all dates of each post
# Finding the topic (should be just one coming from the Listing Page)
li = soup.find("td", {"class": "thead"}).find('strong')
topic = li.text
topic = re.sub("\[\w*\]", '', topic)
topic = topic.replace(",","")
topic = topic.replace("\n","")
topic = soup.find("h1", {"class": "p-title-value"}).text
topic = cleanString(topic.strip()) topic = cleanString(topic.strip())
# Finding the repeated tag that corresponds to the listing of posts
# try:
posts = soup.find('table', {"class": "tborder tfixed clear"}).find('td', {"id": "posts_container"}).find_all(
'div', {"class": "post"})
# For each message (post), get all the fields we are interested to:
for ipost in posts:
# Finding a first level of the HTML page
post_wrapper = ipost.find('span', {"class": "largetext"})
iposts = soup.find('div', {"class": "block-body js-replyNewMessageContainer"}).find_all('article')
# Finding the author (user) of the post
for ipost in iposts:
author = post_wrapper.text.strip()
user.append(cleanString(author)) # Remember to clean the problematic characters
author = ipost.find('h4', {"class": "message-name"}).text
user.append(cleanString(author.strip()))
# Finding the status of the author
stat = ipost.find('h5', {"class": "userTitle message-userTitle"}).text
status.append(cleanString(stat.strip()))
smalltext = ipost.find('div', {"class": "post_author"})
'''
# Testing here two possibilities to find this status and combine them
if ipost.find('div', {"class": "deleted_post_author"}):
status.append(-1)
interest.append(-1)
reputation.append(-1)
addDate.append(-1)
post.append("THIS POST HAS BEEN REMOVED!")
sign.append(-1)
feedback.append(-1)
continue
'''
# CryptBB does have membergroup and postgroup
membergroup = smalltext.find('div', {"class": "profile-rank"})
postgroup = smalltext.find('div', {"class": "postgroup"})
if membergroup != None:
membergroup = membergroup.text.strip()
if postgroup != None:
postgroup = postgroup.text.strip()
membergroup = membergroup + " - " + postgroup
else:
if postgroup != None:
membergroup = postgroup.text.strip()
else:
membergroup = "-1"
status.append(cleanString(membergroup))
# Finding the interest of the author
# CryptBB does not have blurb
blurb = smalltext.find('li', {"class": "blurb"})
if blurb != None:
blurb = blurb.text.strip()
bar = ipost.find('div', {"class": "xtr-progress-bar"})
if bar is not None:
rep = bar.find('p').get('data-value')
else: else:
blurb = "-1"
interest.append(cleanString(blurb))
# Finding the reputation of the user
# CryptBB does have reputation
author_stats = smalltext.find('div', {"class": "author_statistics"})
karma = author_stats.find('strong')
if karma != None:
karma = karma.text
karma = karma.replace("Community Rating: ", "")
karma = karma.replace("Karma: ", "")
karma = karma.strip()
else:
karma = "-1"
reputation.append(cleanString(karma))
# Getting here another good tag to find the post date, post content and users' signature
postarea = ipost.find('div', {"class": "post_content"})
dt = postarea.find('span', {"class": "post_date"}).text
# dt = dt.strip().split()
dt = dt.strip()
day=date.today()
if "Yesterday" in dt:
yesterday = day - timedelta(days=1)
yesterday = yesterday.strftime('%m-%d-%Y')
stime = dt.replace('Yesterday,','').strip()
date_time_obj = yesterday+ ', '+stime
date_time_obj = datetime.strptime(date_time_obj,'%m-%d-%Y, %I:%M %p')
elif "hours ago" in dt:
day = day.strftime('%m-%d-%Y')
date_time_obj = postarea.find('span', {"class": "post_date"}).find('span')['title']
date_time_obj = datetime.strptime(date_time_obj, '%m-%d-%Y, %I:%M %p')
else:
date_time_obj = datetime.strptime(dt, '%m-%d-%Y, %I:%M %p')
stime = date_time_obj.strftime('%b %d, %Y')
sdate = date_time_obj.strftime('%I:%M %p')
addDate.append(date_time_obj)
rep = "-1"
reputation.append(cleanString(rep))
# Finding the post
interest.append("-1")
inner = postarea.find('div', {"class": "post_body scaleimages"})
inner = inner.text.strip()
post.append(cleanString(inner))
# Finding the user's signature
# signature = ipost.find('div', {"class": "post_wrapper"}).find('div', {"class": "moderatorbar"}).find('div', {"class": "signature"})
signature = ipost.find('div', {"class": "signature scaleimages"})
if signature != None:
signature = ipost.find('aside', {"class": "message-signature"})
if signature is not None:
signature = signature.text.strip() signature = signature.text.strip()
# print(signature)
else: else:
signature = "-1" signature = "-1"
sign.append(cleanString(signature)) sign.append(cleanString(signature))
# As no information about user's feedback was found, just assign "-1" to the variable
inner = ipost.find('div', {"class": "bbWrapper"}).find(text=True, recursive=False)
post.append(cleanString(inner.strip()))
feedback.append("-1") feedback.append("-1")
'''
except:
if soup.find('td', {"class": "trow1"}).text == " You do not have permission to access this page. ":
user.append("-1")
status.append(-1)
interest.append(-1)
reputation.append(-1)
addDate.append(-1)
post.append("NO ACCESS TO THIS PAGE!")
sign.append(-1)
feedback.append(-1)
'''
dt = ipost.find('time', {"class": "u-dt"})
date_time_obj = datetime.strptime(dt, '%m-%d-%Y, %I:%M %p')
addDate.append(date_time_obj)
# Populate the final variable (this should be a list with all fields scraped) # Populate the final variable (this should be a list with all fields scraped)
@ -178,74 +72,55 @@ def cryptBB_description_parser(soup):
# This is the method to parse the Listing Pages (one page with many posts) # This is the method to parse the Listing Pages (one page with many posts)
def cryptBB_listing_parser(soup):
board = "-1" # board name (the previous level of the topic in the Forum categorization tree.
# For instance: Security/Malware/Tools to hack Facebook. The board here should be Malware)
nm = 0 # this variable should receive the number of topics
topic = [] # all topics
author = [] # all authors of each topic
views = [] # number of views of each topic
posts = [] # number of posts of each topic
addDate = [] # when the topic was created (difficult to find)
href = [] # this variable should receive all cleaned urls (we will use this to do the marge between
# Listing and Description pages)
# Finding the board (should be just one)
board = soup.find('span', {"class": "active"}).text
def altenens_listing_parser(soup):
nm = 0 # *this variable should receive the number of topics
forum = "Altenens" # 0 *forum name
board = "-1" # 1 *board name (the previous level of the topic in the Forum categorization tree.
# For instance: Security/Malware/Tools to hack Facebook. The board here should be Malware)
author = [] # 2 *all authors of each topic
topic = [] # 3 *all topics
views = [] # 4 number of views of each topic
posts = [] # 5 number of posts of each topic
href = [] # 6 this variable should receive all cleaned urls (we will use this to do the marge between
# Listing and Description pages)
addDate = [] # 7 when the topic was created (difficult to find)
board = soup.find('h1', {"class": "p-title-value"}).text
board = cleanString(board.strip()) board = cleanString(board.strip())
# Finding the repeated tag that corresponds to the listing of topics
itopics = soup.find_all('div', {"class": "structItem-cell structItem-cell--main"})
itopics = soup.find_all('tr',{"class": "inline_row"})
nm = len(itopics)
for itopic in itopics: for itopic in itopics:
# For each topic found, the structure to get the rest of the information can be of two types. Testing all of them
# to don't miss any topic
# Adding the topic to the topic list
try:
topics = itopic.find('span', {"class": "subject_old"}).find('a').text
except:
topics = itopic.find('span', {"class": "subject_new"}).find('a').text
topics = re.sub("\[\w*\]", '', topics)
topic.append(cleanString(topics))
# Counting how many topics we have found so far
topics = itopic.find('div', {"class": "structItem-title"}).text
topic.append(cleanString(topics.strip()))
nm = len(topic)
# Adding the url to the list of urls
try:
link = itopic.find('span', {"class": "subject_old"}).find('a').get('href')
except:
link = itopic.find('span',{"class": "subject_new"}).find('a').get('href')
link = cleanLink(link)
link = itopic.find('a').get('href')
href.append(link) href.append(link)
# Finding the author of the topic
ps = itopic.find('div', {"class":"author smalltext"}).find('a').text
user = ps.strip()
author.append(cleanString(user))
user = itopic.find('div', {"class": "structItem-parts"}).find('a').text
author.append(cleanString(user.strip()))
# Finding the number of replies
columns = itopic.findChildren('td',recursive=False)
replies = columns[3].text
dt = itopic.find('li', {"class": "structItem-startDate"}).get('datetime')
date_time_obj = datetime.strptime(dt, '%m-%d-%Y, %I:%M %p')
addDate.append(date_time_obj)
posts.append(cleanString(replies))
itopics = soup.find_all('div', {"class": "structItem-cell structItem-cell--meta"})
# Finding the number of Views
tview = columns[4].text
views.append(cleanString(tview))
for itopic in itopics:
# If no information about when the topic was added, just assign "-1" to the variable
nposts = itopic.find('dl', {"class": "pairs pairs--justified"}).text
nposts = nposts.replace('K', '000')
posts.append(cleanString(nposts))
addDate.append("-1")
nviews = itopic.find('dl', {"class": "pairs pairs--justified structItem-minor"}).text
nviews = nviews.replace('K', '000')
views.append(cleanString(nviews))
return organizeTopics("CryptBB", nm, topic, board, author, views, posts, href, addDate)
return organizeTopics(forum, nm, board, author, topic, views, posts, href, addDate)
def altenens_links_parser(soup): def altenens_links_parser(soup):
@ -254,10 +129,10 @@ def altenens_links_parser(soup):
href = [] href = []
listing = soup.find('div', {"class": "structItemContainer-group js-threadList"}).find_all('div', {"class": "structItem structItem--thread is-unread js-inlineModContainer js-threadListItem-1843963"})
listing = soup.find_all('div', {"class": "structItem-cell structItem-cell--main"})
for a in listing: for a in listing:
link = a.find('div', {"class": "structItem-title"}).find('a').get('href')
link = a.find('a', {"class": ""}).get('href')
href.append(link) href.append(link)

BIN
Forums/BestCardingWorld/__pycache__/__init__.cpython-311.pyc View File


BIN
Forums/BestCardingWorld/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
Forums/BestCardingWorld/__pycache__/crawler_selenium.cpython-311.pyc View File


BIN
Forums/BestCardingWorld/__pycache__/parser.cpython-310.pyc View File


BIN
Forums/BestCardingWorld/__pycache__/parser.cpython-311.pyc View File


+ 462
- 0
Forums/BestCardingWorld/geckodriver.log View File

@ -0,0 +1,462 @@
1655762162153 geckodriver INFO Listening on 127.0.0.1:50333
1655762166434 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50334" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqUeAAN"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762166904 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50334/devtools/browser/fb880205-35da-44a4-83b8-a861ce7125f1
1655762168590 Marionette INFO Listening on port 50341
1655762169104 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655762207624 Marionette INFO Stopped listening on port 50341
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655762207923 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655762990320 geckodriver INFO Listening on 127.0.0.1:50884
1655762994595 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50885" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileurjxEe"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762995244 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50885/devtools/browser/36503f58-7ab7-4188-81b0-fa1b81f32c0a
1655762996997 Marionette INFO Listening on port 50890
JavaScript error: resource://gre/modules/ExtensionContent.jsm, line 575: TypeError: PrecompiledScript.executeInGlobal: Argument 1 is not an object.
1655762997277 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763083091 Marionette INFO Stopped listening on port 50890
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763083216 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763089557 geckodriver INFO Listening on 127.0.0.1:50923
1655763093759 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50924" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile5mY1qq"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763094208 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50924/devtools/browser/68ce2df9-521f-4258-94f7-c2e2b199701a
1655763095918 Marionette INFO Listening on port 50929
1655763096408 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763143935 Marionette INFO Stopped listening on port 50929
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763144495 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763166975 geckodriver INFO Listening on 127.0.0.1:50947
1655763171175 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50948" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilenAGZM5"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763171719 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50948/devtools/browser/0b9966bd-0cec-45cc-9d11-02a2580233f0
1655763173307 Marionette INFO Listening on port 50953
1655763173368 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763208922 Marionette INFO Stopped listening on port 50953
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763209261 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763387409 geckodriver INFO Listening on 127.0.0.1:49752
1655763391806 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "49753" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilevAHJSJ"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763392827 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:49753/devtools/browser/7fe49a16-940f-4ee8-9a2f-48739a92db78
1655763394700 Marionette INFO Listening on port 49759
1655763395103 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763430848 Marionette INFO Stopped listening on port 49759
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763431145 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655765589286 geckodriver INFO Listening on 127.0.0.1:50102
1655765621677 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50103" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ecUpb"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655765622149 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50103/devtools/browser/e5a916fc-987c-43d4-9c12-9c9b88cca242
1655765623776 Marionette INFO Listening on port 50109
1655765623892 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766125168 Marionette INFO Stopped listening on port 50109
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655766125303 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655766505099 geckodriver INFO Listening on 127.0.0.1:50156
1655766509351 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50157" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileVqvmzh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655766509876 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50157/devtools/browser/13084c14-64d3-48a5-8b99-4c514e961d4f
1655766511328 Marionette INFO Listening on port 50162
1655766511395 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766552298 Marionette INFO Stopped listening on port 50162
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655766552578 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767657126 geckodriver INFO Listening on 127.0.0.1:50257
1655767661375 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50258" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileB9Dzeh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767661843 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50258/devtools/browser/867cd318-6430-4b3f-9b63-8c6bdf17636a
1655767663330 Marionette INFO Listening on port 50263
1655767663431 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767668263 Marionette INFO Stopped listening on port 50263
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655767668591 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767689480 geckodriver INFO Listening on 127.0.0.1:50276
1655767693635 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50277" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ZOQe7"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767694215 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50277/devtools/browser/a70a3f5b-19af-4a68-99c8-c46086ba9599
1655767695725 Marionette INFO Listening on port 50282
1655767695830 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767701272 Marionette INFO Stopped listening on port 50282
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767701594 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767791360 geckodriver INFO Listening on 127.0.0.1:50296
1655767795575 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50297" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile4GEqBr"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767796001 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50297/devtools/browser/e2d4e57a-16ce-429d-9804-e9e391199ddf
1655767797534 Marionette INFO Listening on port 50302
1655767797661 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767802833 Marionette INFO Stopped listening on port 50302
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767803151 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767902908 geckodriver INFO Listening on 127.0.0.1:50320
1655767907009 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50321" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqlJ8ZA"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767907484 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50321/devtools/browser/49e77669-01dd-447e-a804-36d42b3400cf
1655767908782 Marionette INFO Listening on port 50326
1655767909069 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767916218 Marionette INFO Stopped listening on port 50326
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767916523 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767958348 geckodriver INFO Listening on 127.0.0.1:50340
1655767962557 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50341" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilekpCdfs"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767963032 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50341/devtools/browser/b4d1728c-d26f-4537-adc2-5c954b171b13
1655767964591 Marionette INFO Listening on port 50346
1655767964633 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767973210 Marionette INFO Stopped listening on port 50346
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767973501 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767998084 geckodriver INFO Listening on 127.0.0.1:50358
1655768002314 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50359" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileujYNBj"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768002757 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50359/devtools/browser/04112c45-204d-43f1-9615-34782fd06632
1655768004165 Marionette INFO Listening on port 50364
1655768004375 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768009787 Marionette INFO Stopped listening on port 50364
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655768010100 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768083966 geckodriver INFO Listening on 127.0.0.1:50386
1655768088234 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50387" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile39Pl1f"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768088664 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50387/devtools/browser/3d6cda42-cb60-4c67-8d76-1d186a51887f
1655768090263 Marionette INFO Listening on port 50392
1655768090299 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768091655 Marionette INFO Stopped listening on port 50392
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
1655768092659 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768101201 geckodriver INFO Listening on 127.0.0.1:50404
1655768105373 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50405" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile2YRGSz"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768105853 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50405/devtools/browser/9c4a4d71-25ff-4980-a54d-8545d6200790
1655768107431 Marionette INFO Listening on port 50410
1655768107530 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768119155 Marionette INFO Stopped listening on port 50410
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655768119438 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768229472 geckodriver INFO Listening on 127.0.0.1:50431
1655768233458 webdriver::server WARN Rejected request with missing Host header

+ 331
- 0
Forums/Cardingleaks/crawler_selenium.py View File

@ -0,0 +1,331 @@
__author__ = 'DarkWeb'
'''
Cardingleaks Forum Crawler (Selenium)
FIXED
'''
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from PIL import Image
import urllib.parse as urlparse
import os, re, time
import subprocess
from bs4 import BeautifulSoup
from Forums.Initialization.prepare_parser import new_parse
from Forums.Cardingleaks.parser import cardingleaks_links_parser
from Forums.Utilities.utilities import cleanHTML
counter = 1
baseURL = 'https://cardingleaks.ws/'
# Opens Tor Browser, crawls the website
def startCrawling():
opentor()
forumName = getForumName()
driver = getAccess()
if driver != 'down':
try:
login(driver)
crawlForum(driver)
except Exception as e:
print(driver.current_url, e)
closetor(driver)
new_parse(forumName, baseURL, False)
# Opens Tor Browser
def opentor():
from Forums.Initialization.forums_mining import config
global pid
print("Connecting Tor...")
pro = subprocess.Popen(config.get('TOR', 'firefox_binary_path'))
pid = pro.pid
time.sleep(7.5)
input('Tor Connected. Press ENTER to continue\n')
return
# Login using premade account credentials and do login captcha manually
def login(driver):
#click login button
login_link = driver.find_element(
by=By.XPATH, value='/html/body/div[2]/div[1]/nav/div/div[3]/div[1]/a[1]').\
get_attribute('href')
driver.get(login_link)# open tab with url
#entering username and password into input boxes
usernameBox = driver.find_element(by=By.NAME, value='login')
#Username here
usernameBox.send_keys('somanyfrogs')#sends string to the username box
passwordBox = driver.find_element(by=By.NAME, value='password')
#Password here
passwordBox.send_keys('therearewaytoomanyherehowwhy')# sends string to passwordBox
login = driver.find_element(by=By.CLASS_NAME, value='block-container')
login_link = login.find_element(by=By.TAG_NAME, value='button')
login_link.click()
# input('input')
# wait for listing page show up (This Xpath may need to change based on different seed url)
# wait for 50 sec until id = tab_content is found, then cont
WebDriverWait(driver, 50).until(EC.visibility_of_element_located(
(By.CLASS_NAME, 'p-body-pageContent')))
# Returns the name of the website
def getForumName() -> str:
name = 'Cardingleaks'
return name
# Return the link of the website
def getFixedURL():
url = 'https://cardingleaks.ws/'
return url
# Closes Tor Browser
def closetor(driver):
# global pid
# os.system("taskkill /pid " + str(pro.pid))
# os.system("taskkill /t /f /im tor.exe")
print('Closing Tor...')
driver.close() #close tab
time.sleep(3)
return
# Creates FireFox 'driver' and configure its 'Profile'
# to use Tor proxy and socket
def createFFDriver():
from Forums.Initialization.forums_mining import config
ff_binary = FirefoxBinary(config.get('TOR', 'firefox_binary_path'))
ff_prof = FirefoxProfile(config.get('TOR', 'firefox_profile_path'))
ff_prof.set_preference("places.history.enabled", False)
ff_prof.set_preference("privacy.clearOnShutdown.offlineApps", True)
ff_prof.set_preference("privacy.clearOnShutdown.passwords", True)
ff_prof.set_preference("privacy.clearOnShutdown.siteSettings", True)
ff_prof.set_preference("privacy.sanitize.sanitizeOnShutdown", True)
ff_prof.set_preference("signon.rememberSignons", False)
ff_prof.set_preference("network.cookie.lifetimePolicy", 2)
ff_prof.set_preference("network.dns.disablePrefetch", True)
ff_prof.set_preference("network.http.sendRefererHeader", 0)
ff_prof.set_preference("permissions.default.image", 3)
ff_prof.set_preference("browser.download.folderList", 2)
ff_prof.set_preference("browser.download.manager.showWhenStarting", False)
ff_prof.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/plain")
ff_prof.set_preference('network.proxy.type', 1)
ff_prof.set_preference("network.proxy.socks_version", 5)
ff_prof.set_preference('network.proxy.socks', '127.0.0.1')
ff_prof.set_preference('network.proxy.socks_port', 9150)
ff_prof.set_preference('network.proxy.socks_remote_dns', True)
ff_prof.set_preference("javascript.enabled", True)
ff_prof.update_preferences()
service = Service(config.get('TOR', 'geckodriver_path'))
driver = webdriver.Firefox(firefox_binary=ff_binary, firefox_profile=ff_prof, service=service)
return driver
def getAccess():
url = getFixedURL()
driver = createFFDriver()
try:
driver.get(url)
return driver
except:
driver.close()
return 'down'
# Saves the crawled html page
def savePage(page, url):
cleanPage = cleanHTML(page)
filePath = getFullPathName(url)
os.makedirs(os.path.dirname(filePath), exist_ok=True)
open(filePath, 'wb').write(cleanPage.encode('utf-8'))
return
# Gets the full path of the page to be saved along with its appropriate file name
def getFullPathName(url):
from Forums.Initialization.forums_mining import config, CURRENT_DATE
mainDir = os.path.join(config.get('Project', 'shared_folder'), "Forums/" + getForumName() + "/HTML_Pages")
fileName = getNameFromURL(url)
if isDescriptionLink(url):
fullPath = os.path.join(mainDir, CURRENT_DATE + r'\\Description\\' + fileName + '.html')
else:
fullPath = os.path.join(mainDir, CURRENT_DATE + r'\\Listing\\' + fileName + '.html')
return fullPath
# Creates the file name from passed URL
def getNameFromURL(url):
global counter
name = ''.join(e for e in url if e.isalnum())
if name == '':
name = str(counter)
counter = counter + 1
return name
def getInterestedLinks():
links = []
# # carding methods
links.append('https://cardingleaks.ws/forums/carding-methods.82/')
# # carding schools
# links.append('https://cardingleaks.ws/forums/help-desk-carding-school.35/')
# # carding discussion
# links.append('https://cardingleaks.ws/forums/carding-discussion-desk.58/')
# # carding tutorials
# links.append('https://cardingleaks.ws/forums/carding-tutorials.13/')
# # carding tools and software
# links.append('https://cardingleaks.ws/forums/carding-tools-softwares.10/')
# # exploits and cracking tools
# links.append('https://cardingleaks.ws/forums/exploits-cracking-tools.22/')
return links
def crawlForum(driver):
print("Crawling the Cardingleaks forum")
linksToCrawl = getInterestedLinks()
visited = set(linksToCrawl)
initialTime = time.time()
i = 0
count = 0
while i < len(linksToCrawl):
link = linksToCrawl[i]
print('Crawling :', link)
try:
try:
driver.get(link)
except:
driver.refresh()
html = driver.page_source
savePage(html, link)
has_next_page = True
while has_next_page:
list = topicPages(html)
for item in list:
itemURL = urlparse.urljoin(baseURL, str(item))
try:
driver.get(itemURL)
except:
driver.refresh()
savePage(driver.page_source, item)
driver.back()
#variable to check if there is a next page for the topic
has_next_topic_page = True
counter = 1
# check if there is a next page for the topics
while has_next_topic_page:
# try to access next page of th topic
itemURL = urlparse.urljoin(baseURL, str(item))
try:
driver.get(itemURL)
except:
driver.refresh()
savePage(driver.page_source, item)
# if there is a next page then go and save....
# Spec
try:
# temp = driver.find_element(By.XPATH, '/html/body/div[2]/div[4]/div/div[5]/div[2]/div/div[1]/div[1]/div/nav/div[1]') # /html/body/div/div[2]/div/div[2]/div/
item = driver.find_element(by=By.LINK_TEXT, value='Next').get_attribute('href') #/html/body/div/div[2]/div/div[2]/div
if item == "":
raise NoSuchElementException
else:
counter += 1
except NoSuchElementException:
has_next_topic_page = False
# end of loop
for i in range(counter):
driver.back()
# comment out
break
# comment out
if count == 1:
count = 0
break
try:
# temp = driver.find_element(by=By.XPATH, value = '/html/body/div[2]/div[4]/div/div[5]/div[2]/div/div/div[1]/div/nav/div[1]')
link = driver.find_element(by=By.LINK_TEXT, value='Next').get_attribute('href')
if link == "":
raise NoSuchElementException
try:
driver.get(link)
except:
driver.refresh()
html = driver.page_source
savePage(html, link)
count += 1
except NoSuchElementException:
has_next_page = False
except Exception as e:
print(link, e)
i += 1
# finalTime = time.time()
# print finalTime - initialTime
input("Crawling Cardingleaks forum done successfully. Press ENTER to continue\n")
# Returns 'True' if the link is Topic link, may need to change for every website
def isDescriptionLink(url):
if 'threads' in url:
return True
return False
# Returns True if the link is a listingPage link, may need to change for every website
def isListingLink(url):
if 'forums' in url:
return True
return False
# calling the parser to define the links
def topicPages(html):
soup = BeautifulSoup(html, "html.parser")
return cardingleaks_links_parser(soup)
def crawler():
startCrawling()
# print("Crawling and Parsing BestCardingWorld .... DONE!")

+ 247
- 0
Forums/Cardingleaks/parser.py View File

@ -0,0 +1,247 @@
__author__ = 'DarkWeb'
# Here, we are importing the auxiliary functions to clean or convert data
from Forums.Utilities.utilities import *
from datetime import date
from datetime import timedelta
import re
# Here, we are importing BeautifulSoup to search through the HTML tree
from bs4 import BeautifulSoup
# This is the method to parse the Description Pages (one page to each topic in the Listing Pages)
def cryptBB_description_parser(soup):
# Fields to be parsed
topic = "-1" # 0 *topic name
user = [] # 1 *all users of each post
status = [] # 2 all user's authority in each post such as (adm, member, dangerous)
reputation = [] # 3 all user's karma in each post (usually found as a number)
interest = [] # 4 all user's interest in each post
sign = [] # 5 all user's signature in each post (usually a standard message after the content of the post)
post = [] # 6 all messages of each post
feedback = [] # 7 all feedbacks of each vendor (this was found in just one Forum and with a number format)
addDate = [] # 8 all dates of each post
# Finding the topic (should be just one coming from the Listing Page)
li = soup.find("td", {"class": "thead"}).find('strong')
topic = li.text
topic = re.sub("\[\w*\]", '', topic)
topic = topic.replace(",","")
topic = topic.replace("\n","")
topic = cleanString(topic.strip())
# Finding the repeated tag that corresponds to the listing of posts
# try:
posts = soup.find('table', {"class": "tborder tfixed clear"}).find('td', {"id": "posts_container"}).find_all(
'div', {"class": "post"})
# For each message (post), get all the fields we are interested to:
for ipost in posts:
# Finding a first level of the HTML page
post_wrapper = ipost.find('span', {"class": "largetext"})
# Finding the author (user) of the post
author = post_wrapper.text.strip()
user.append(cleanString(author)) # Remember to clean the problematic characters
# Finding the status of the author
smalltext = ipost.find('div', {"class": "post_author"})
'''
# Testing here two possibilities to find this status and combine them
if ipost.find('div', {"class": "deleted_post_author"}):
status.append(-1)
interest.append(-1)
reputation.append(-1)
addDate.append(-1)
post.append("THIS POST HAS BEEN REMOVED!")
sign.append(-1)
feedback.append(-1)
continue
'''
# CryptBB does have membergroup and postgroup
membergroup = smalltext.find('div', {"class": "profile-rank"})
postgroup = smalltext.find('div', {"class": "postgroup"})
if membergroup != None:
membergroup = membergroup.text.strip()
if postgroup != None:
postgroup = postgroup.text.strip()
membergroup = membergroup + " - " + postgroup
else:
if postgroup != None:
membergroup = postgroup.text.strip()
else:
membergroup = "-1"
status.append(cleanString(membergroup))
# Finding the interest of the author
# CryptBB does not have blurb
blurb = smalltext.find('li', {"class": "blurb"})
if blurb != None:
blurb = blurb.text.strip()
else:
blurb = "-1"
interest.append(cleanString(blurb))
# Finding the reputation of the user
# CryptBB does have reputation
author_stats = smalltext.find('div', {"class": "author_statistics"})
karma = author_stats.find('strong')
if karma != None:
karma = karma.text
karma = karma.replace("Community Rating: ", "")
karma = karma.replace("Karma: ", "")
karma = karma.strip()
else:
karma = "-1"
reputation.append(cleanString(karma))
# Getting here another good tag to find the post date, post content and users' signature
postarea = ipost.find('div', {"class": "post_content"})
dt = postarea.find('span', {"class": "post_date"}).text
# dt = dt.strip().split()
dt = dt.strip()
day=date.today()
if "Yesterday" in dt:
yesterday = day - timedelta(days=1)
yesterday = yesterday.strftime('%m-%d-%Y')
stime = dt.replace('Yesterday,','').strip()
date_time_obj = yesterday+ ', '+stime
date_time_obj = datetime.strptime(date_time_obj,'%m-%d-%Y, %I:%M %p')
elif "hours ago" in dt:
day = day.strftime('%m-%d-%Y')
date_time_obj = postarea.find('span', {"class": "post_date"}).find('span')['title']
date_time_obj = datetime.strptime(date_time_obj, '%m-%d-%Y, %I:%M %p')
else:
date_time_obj = datetime.strptime(dt, '%m-%d-%Y, %I:%M %p')
stime = date_time_obj.strftime('%b %d, %Y')
sdate = date_time_obj.strftime('%I:%M %p')
addDate.append(date_time_obj)
# Finding the post
inner = postarea.find('div', {"class": "post_body scaleimages"})
inner = inner.text.strip()
post.append(cleanString(inner))
# Finding the user's signature
# signature = ipost.find('div', {"class": "post_wrapper"}).find('div', {"class": "moderatorbar"}).find('div', {"class": "signature"})
signature = ipost.find('div', {"class": "signature scaleimages"})
if signature != None:
signature = signature.text.strip()
# print(signature)
else:
signature = "-1"
sign.append(cleanString(signature))
# As no information about user's feedback was found, just assign "-1" to the variable
feedback.append("-1")
# Populate the final variable (this should be a list with all fields scraped)
row = (topic, user, status, reputation, interest, sign, post, feedback, addDate)
# Sending the results
return row
# This is the method to parse the Listing Pages (one page with many posts)
def cryptBB_listing_parser(soup):
nm = 0 # *this variable should receive the number of topics
forum = "OnniForums" # 0 *forum name
board = "-1" # 1 *board name (the previous level of the topic in the Forum categorization tree.
# For instance: Security/Malware/Tools to hack Facebook. The board here should be Malware)
author = [] # 2 *all authors of each topic
topic = [] # 3 *all topics
views = [] # 4 number of views of each topic
posts = [] # 5 number of posts of each topic
href = [] # 6 this variable should receive all cleaned urls (we will use this to do the marge between
# Listing and Description pages)
addDate = [] # 7 when the topic was created (difficult to find)
# Finding the board (should be just one)
board = soup.find('span', {"class": "active"}).text
board = cleanString(board.strip())
# Finding the repeated tag that corresponds to the listing of topics
itopics = soup.find_all('tr',{"class": "inline_row"})
for itopic in itopics:
# For each topic found, the structure to get the rest of the information can be of two types. Testing all of them
# to don't miss any topic
# Adding the topic to the topic list
try:
topics = itopic.find('span', {"class": "subject_old"}).find('a').text
except:
topics = itopic.find('span', {"class": "subject_new"}).find('a').text
topics = re.sub("\[\w*\]", '', topics)
topic.append(cleanString(topics))
# Counting how many topics we have found so far
nm = len(topic)
# Adding the url to the list of urls
try:
link = itopic.find('span', {"class": "subject_old"}).find('a').get('href')
except:
link = itopic.find('span',{"class": "subject_new"}).find('a').get('href')
href.append(link)
# Finding the author of the topic
ps = itopic.find('div', {"class":"author smalltext"}).find('a').text
user = ps.strip()
author.append(cleanString(user))
# Finding the number of replies
columns = itopic.findChildren('td',recursive=False)
replies = columns[3].text
posts.append(cleanString(replies))
# Finding the number of Views
tview = columns[4].text
views.append(cleanString(tview))
# If no information about when the topic was added, just assign "-1" to the variable
addDate.append("-1")
return organizeTopics(forum, nm, board, author, topic, views, posts, href, addDate)
def cardingleaks_links_parser(soup):
# Returning all links that should be visited by the Crawler
href = []
listing = soup.find_all('div', {"class": "structItem-title"})
for a in listing:
link = a.find('a').get('href')
href.append(link)
return href

BIN
Forums/Classifier/__pycache__/__init__.cpython-310.pyc View File


BIN
Forums/Classifier/__pycache__/__init__.cpython-311.pyc View File


BIN
Forums/Classifier/__pycache__/classify_product.cpython-310.pyc View File


BIN
Forums/Classifier/__pycache__/classify_product.cpython-311.pyc View File


BIN
Forums/Classifier/__pycache__/transformer.cpython-310.pyc View File


BIN
Forums/Classifier/__pycache__/transformer.cpython-311.pyc View File


BIN
Forums/CryptBB/__pycache__/__init__.cpython-311.pyc View File


BIN
Forums/CryptBB/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
Forums/CryptBB/__pycache__/crawler_selenium.cpython-311.pyc View File


BIN
Forums/CryptBB/__pycache__/parser.cpython-310.pyc View File


BIN
Forums/CryptBB/__pycache__/parser.cpython-311.pyc View File


BIN
Forums/CryptBB/captcha.png View File

Before After
Width: 200  |  Height: 60  |  Size: 16 KiB

+ 462
- 0
Forums/CryptBB/geckodriver.log View File

@ -0,0 +1,462 @@
1655762162153 geckodriver INFO Listening on 127.0.0.1:50333
1655762166434 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50334" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqUeAAN"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762166904 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50334/devtools/browser/fb880205-35da-44a4-83b8-a861ce7125f1
1655762168590 Marionette INFO Listening on port 50341
1655762169104 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655762207624 Marionette INFO Stopped listening on port 50341
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655762207923 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655762990320 geckodriver INFO Listening on 127.0.0.1:50884
1655762994595 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50885" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileurjxEe"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762995244 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50885/devtools/browser/36503f58-7ab7-4188-81b0-fa1b81f32c0a
1655762996997 Marionette INFO Listening on port 50890
JavaScript error: resource://gre/modules/ExtensionContent.jsm, line 575: TypeError: PrecompiledScript.executeInGlobal: Argument 1 is not an object.
1655762997277 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763083091 Marionette INFO Stopped listening on port 50890
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763083216 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763089557 geckodriver INFO Listening on 127.0.0.1:50923
1655763093759 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50924" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile5mY1qq"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763094208 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50924/devtools/browser/68ce2df9-521f-4258-94f7-c2e2b199701a
1655763095918 Marionette INFO Listening on port 50929
1655763096408 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763143935 Marionette INFO Stopped listening on port 50929
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763144495 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763166975 geckodriver INFO Listening on 127.0.0.1:50947
1655763171175 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50948" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilenAGZM5"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763171719 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50948/devtools/browser/0b9966bd-0cec-45cc-9d11-02a2580233f0
1655763173307 Marionette INFO Listening on port 50953
1655763173368 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763208922 Marionette INFO Stopped listening on port 50953
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763209261 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763387409 geckodriver INFO Listening on 127.0.0.1:49752
1655763391806 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "49753" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilevAHJSJ"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763392827 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:49753/devtools/browser/7fe49a16-940f-4ee8-9a2f-48739a92db78
1655763394700 Marionette INFO Listening on port 49759
1655763395103 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763430848 Marionette INFO Stopped listening on port 49759
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763431145 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655765589286 geckodriver INFO Listening on 127.0.0.1:50102
1655765621677 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50103" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ecUpb"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655765622149 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50103/devtools/browser/e5a916fc-987c-43d4-9c12-9c9b88cca242
1655765623776 Marionette INFO Listening on port 50109
1655765623892 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766125168 Marionette INFO Stopped listening on port 50109
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655766125303 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655766505099 geckodriver INFO Listening on 127.0.0.1:50156
1655766509351 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50157" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileVqvmzh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655766509876 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50157/devtools/browser/13084c14-64d3-48a5-8b99-4c514e961d4f
1655766511328 Marionette INFO Listening on port 50162
1655766511395 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766552298 Marionette INFO Stopped listening on port 50162
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655766552578 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767657126 geckodriver INFO Listening on 127.0.0.1:50257
1655767661375 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50258" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileB9Dzeh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767661843 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50258/devtools/browser/867cd318-6430-4b3f-9b63-8c6bdf17636a
1655767663330 Marionette INFO Listening on port 50263
1655767663431 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767668263 Marionette INFO Stopped listening on port 50263
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655767668591 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767689480 geckodriver INFO Listening on 127.0.0.1:50276
1655767693635 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50277" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ZOQe7"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767694215 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50277/devtools/browser/a70a3f5b-19af-4a68-99c8-c46086ba9599
1655767695725 Marionette INFO Listening on port 50282
1655767695830 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767701272 Marionette INFO Stopped listening on port 50282
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767701594 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767791360 geckodriver INFO Listening on 127.0.0.1:50296
1655767795575 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50297" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile4GEqBr"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767796001 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50297/devtools/browser/e2d4e57a-16ce-429d-9804-e9e391199ddf
1655767797534 Marionette INFO Listening on port 50302
1655767797661 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767802833 Marionette INFO Stopped listening on port 50302
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767803151 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767902908 geckodriver INFO Listening on 127.0.0.1:50320
1655767907009 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50321" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqlJ8ZA"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767907484 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50321/devtools/browser/49e77669-01dd-447e-a804-36d42b3400cf
1655767908782 Marionette INFO Listening on port 50326
1655767909069 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767916218 Marionette INFO Stopped listening on port 50326
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767916523 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767958348 geckodriver INFO Listening on 127.0.0.1:50340
1655767962557 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50341" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilekpCdfs"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767963032 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50341/devtools/browser/b4d1728c-d26f-4537-adc2-5c954b171b13
1655767964591 Marionette INFO Listening on port 50346
1655767964633 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767973210 Marionette INFO Stopped listening on port 50346
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767973501 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767998084 geckodriver INFO Listening on 127.0.0.1:50358
1655768002314 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50359" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileujYNBj"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768002757 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50359/devtools/browser/04112c45-204d-43f1-9615-34782fd06632
1655768004165 Marionette INFO Listening on port 50364
1655768004375 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768009787 Marionette INFO Stopped listening on port 50364
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655768010100 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768083966 geckodriver INFO Listening on 127.0.0.1:50386
1655768088234 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50387" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile39Pl1f"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768088664 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50387/devtools/browser/3d6cda42-cb60-4c67-8d76-1d186a51887f
1655768090263 Marionette INFO Listening on port 50392
1655768090299 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768091655 Marionette INFO Stopped listening on port 50392
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
1655768092659 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768101201 geckodriver INFO Listening on 127.0.0.1:50404
1655768105373 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50405" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile2YRGSz"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768105853 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50405/devtools/browser/9c4a4d71-25ff-4980-a54d-8545d6200790
1655768107431 Marionette INFO Listening on port 50410
1655768107530 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768119155 Marionette INFO Stopped listening on port 50410
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655768119438 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768229472 geckodriver INFO Listening on 127.0.0.1:50431
1655768233458 webdriver::server WARN Rejected request with missing Host header

+ 1
- 1
Forums/CryptBB/parser.py View File

@ -168,7 +168,7 @@ def cryptBB_description_parser(soup):
def cryptBB_listing_parser(soup): def cryptBB_listing_parser(soup):
nm = 0 # *this variable should receive the number of topics nm = 0 # *this variable should receive the number of topics
forum = "OnniForums" # 0 *forum name
forum = "CryptBB" # 0 *forum name
board = "-1" # 1 *board name (the previous level of the topic in the Forum categorization tree. board = "-1" # 1 *board name (the previous level of the topic in the Forum categorization tree.
# For instance: Security/Malware/Tools to hack Facebook. The board here should be Malware) # For instance: Security/Malware/Tools to hack Facebook. The board here should be Malware)
author = [] # 2 *all authors of each topic author = [] # 2 *all authors of each topic


BIN
Forums/DB_Connection/__pycache__/__init__.cpython-310.pyc View File


BIN
Forums/DB_Connection/__pycache__/__init__.cpython-311.pyc View File


BIN
Forums/DB_Connection/__pycache__/db_connection.cpython-310.pyc View File


BIN
Forums/DB_Connection/__pycache__/db_connection.cpython-311.pyc View File


BIN
Forums/HiddenAnswers/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
Forums/HiddenAnswers/__pycache__/crawler_selenium.cpython-311.pyc View File


BIN
Forums/HiddenAnswers/__pycache__/parser.cpython-310.pyc View File


BIN
Forums/HiddenAnswers/__pycache__/parser.cpython-311.pyc View File


+ 462
- 0
Forums/HiddenAnswers/geckodriver.log View File

@ -0,0 +1,462 @@
1655762162153 geckodriver INFO Listening on 127.0.0.1:50333
1655762166434 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50334" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqUeAAN"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762166904 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50334/devtools/browser/fb880205-35da-44a4-83b8-a861ce7125f1
1655762168590 Marionette INFO Listening on port 50341
1655762169104 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655762207624 Marionette INFO Stopped listening on port 50341
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655762207923 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655762990320 geckodriver INFO Listening on 127.0.0.1:50884
1655762994595 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50885" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileurjxEe"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762995244 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50885/devtools/browser/36503f58-7ab7-4188-81b0-fa1b81f32c0a
1655762996997 Marionette INFO Listening on port 50890
JavaScript error: resource://gre/modules/ExtensionContent.jsm, line 575: TypeError: PrecompiledScript.executeInGlobal: Argument 1 is not an object.
1655762997277 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763083091 Marionette INFO Stopped listening on port 50890
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763083216 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763089557 geckodriver INFO Listening on 127.0.0.1:50923
1655763093759 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50924" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile5mY1qq"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763094208 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50924/devtools/browser/68ce2df9-521f-4258-94f7-c2e2b199701a
1655763095918 Marionette INFO Listening on port 50929
1655763096408 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763143935 Marionette INFO Stopped listening on port 50929
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763144495 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763166975 geckodriver INFO Listening on 127.0.0.1:50947
1655763171175 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50948" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilenAGZM5"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763171719 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50948/devtools/browser/0b9966bd-0cec-45cc-9d11-02a2580233f0
1655763173307 Marionette INFO Listening on port 50953
1655763173368 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763208922 Marionette INFO Stopped listening on port 50953
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763209261 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763387409 geckodriver INFO Listening on 127.0.0.1:49752
1655763391806 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "49753" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilevAHJSJ"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763392827 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:49753/devtools/browser/7fe49a16-940f-4ee8-9a2f-48739a92db78
1655763394700 Marionette INFO Listening on port 49759
1655763395103 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763430848 Marionette INFO Stopped listening on port 49759
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763431145 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655765589286 geckodriver INFO Listening on 127.0.0.1:50102
1655765621677 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50103" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ecUpb"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655765622149 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50103/devtools/browser/e5a916fc-987c-43d4-9c12-9c9b88cca242
1655765623776 Marionette INFO Listening on port 50109
1655765623892 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766125168 Marionette INFO Stopped listening on port 50109
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655766125303 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655766505099 geckodriver INFO Listening on 127.0.0.1:50156
1655766509351 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50157" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileVqvmzh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655766509876 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50157/devtools/browser/13084c14-64d3-48a5-8b99-4c514e961d4f
1655766511328 Marionette INFO Listening on port 50162
1655766511395 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766552298 Marionette INFO Stopped listening on port 50162
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655766552578 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767657126 geckodriver INFO Listening on 127.0.0.1:50257
1655767661375 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50258" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileB9Dzeh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767661843 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50258/devtools/browser/867cd318-6430-4b3f-9b63-8c6bdf17636a
1655767663330 Marionette INFO Listening on port 50263
1655767663431 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767668263 Marionette INFO Stopped listening on port 50263
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655767668591 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767689480 geckodriver INFO Listening on 127.0.0.1:50276
1655767693635 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50277" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ZOQe7"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767694215 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50277/devtools/browser/a70a3f5b-19af-4a68-99c8-c46086ba9599
1655767695725 Marionette INFO Listening on port 50282
1655767695830 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767701272 Marionette INFO Stopped listening on port 50282
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767701594 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767791360 geckodriver INFO Listening on 127.0.0.1:50296
1655767795575 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50297" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile4GEqBr"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767796001 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50297/devtools/browser/e2d4e57a-16ce-429d-9804-e9e391199ddf
1655767797534 Marionette INFO Listening on port 50302
1655767797661 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767802833 Marionette INFO Stopped listening on port 50302
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767803151 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767902908 geckodriver INFO Listening on 127.0.0.1:50320
1655767907009 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50321" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqlJ8ZA"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767907484 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50321/devtools/browser/49e77669-01dd-447e-a804-36d42b3400cf
1655767908782 Marionette INFO Listening on port 50326
1655767909069 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767916218 Marionette INFO Stopped listening on port 50326
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767916523 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767958348 geckodriver INFO Listening on 127.0.0.1:50340
1655767962557 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50341" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilekpCdfs"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767963032 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50341/devtools/browser/b4d1728c-d26f-4537-adc2-5c954b171b13
1655767964591 Marionette INFO Listening on port 50346
1655767964633 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767973210 Marionette INFO Stopped listening on port 50346
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767973501 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767998084 geckodriver INFO Listening on 127.0.0.1:50358
1655768002314 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50359" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileujYNBj"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768002757 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50359/devtools/browser/04112c45-204d-43f1-9615-34782fd06632
1655768004165 Marionette INFO Listening on port 50364
1655768004375 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768009787 Marionette INFO Stopped listening on port 50364
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655768010100 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768083966 geckodriver INFO Listening on 127.0.0.1:50386
1655768088234 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50387" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile39Pl1f"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768088664 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50387/devtools/browser/3d6cda42-cb60-4c67-8d76-1d186a51887f
1655768090263 Marionette INFO Listening on port 50392
1655768090299 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768091655 Marionette INFO Stopped listening on port 50392
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
1655768092659 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768101201 geckodriver INFO Listening on 127.0.0.1:50404
1655768105373 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50405" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile2YRGSz"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768105853 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50405/devtools/browser/9c4a4d71-25ff-4980-a54d-8545d6200790
1655768107431 Marionette INFO Listening on port 50410
1655768107530 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768119155 Marionette INFO Stopped listening on port 50410
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655768119438 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768229472 geckodriver INFO Listening on 127.0.0.1:50431
1655768233458 webdriver::server WARN Rejected request with missing Host header

BIN
Forums/Initialization/__pycache__/__init__.cpython-310.pyc View File


BIN
Forums/Initialization/__pycache__/__init__.cpython-311.pyc View File


BIN
Forums/Initialization/__pycache__/forums_mining.cpython-310.pyc View File


BIN
Forums/Initialization/__pycache__/forums_mining.cpython-311.pyc View File


BIN
Forums/Initialization/__pycache__/prepare_parser.cpython-310.pyc View File


BIN
Forums/Initialization/__pycache__/prepare_parser.cpython-311.pyc View File


+ 9
- 6
Forums/Initialization/forums_mining.py View File

@ -1,7 +1,7 @@
__author__ = 'DarkWeb' __author__ = 'DarkWeb'
''' '''
Starting point of the Darkweb Mining Platform
''' '''
import os import os
@ -10,8 +10,10 @@ from Forums.BestCardingWorld.crawler_selenium import crawler as crawlerBestCardi
from Forums.CryptBB.crawler_selenium import crawler as crawlerCryptBB from Forums.CryptBB.crawler_selenium import crawler as crawlerCryptBB
from Forums.OnniForums.crawler_selenium import crawler as crawlerOnniForums from Forums.OnniForums.crawler_selenium import crawler as crawlerOnniForums
from Forums.AbyssForum.crawler_selenium import crawler as crawlerAbyssForum from Forums.AbyssForum.crawler_selenium import crawler as crawlerAbyssForum
from Forums.Altenens.crawler_selenium import crawler as crawlerAltenensForum
from Forums.Procrax.crawler_selenium import crawler as crawlerProcraxForum
from Forums.HiddenAnswers.crawler_selenium import crawler as crawlerHiddenAnswers from Forums.HiddenAnswers.crawler_selenium import crawler as crawlerHiddenAnswers
from Forums.Cardingleaks.crawler_selenium import crawler as crawlerCardingleaks
from Forums.Altenens.crawler_selenium import crawler as crawlerAltenens
import configparser import configparser
import time import time
@ -112,10 +114,11 @@ if __name__ == '__main__':
elif forum == "HiddenAnswers": elif forum == "HiddenAnswers":
crawlerHiddenAnswers() crawlerHiddenAnswers()
elif forum == "Altenens": elif forum == "Altenens":
crawlerAltenensForum()
crawlerAltenens()
elif forum == 'Procrax':
crawlerProcraxForum()
elif forum == 'Cardingleaks':
crawlerCardingleaks()


+ 10953
- 0
Forums/Initialization/geckodriver.log
File diff suppressed because it is too large
View File


+ 330
- 0
Forums/Libre/crawler_selenium.py View File

@ -0,0 +1,330 @@
__author__ = 'DarkWeb'
'''
Libre Forum Crawler (Selenium)
'''
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from PIL import Image
import urllib.parse as urlparse
import os, re, time
import subprocess
from bs4 import BeautifulSoup
from Forums.Initialization.prepare_parser import new_parse
from Forums.Libre.parser import libre_links_parser
from Forums.Utilities.utilities import cleanHTML
counter = 1
baseURL = 'http://libreeunomyly6ot7kspglmbd5cvlkogib6rozy43r2glatc6rmwauqd.onion/'
# Opens Tor Browser, crawls the website
def startCrawling():
opentor()
forumName = getForumName()
driver = getAccess()
if driver != 'down':
try:
login(driver)
crawlForum(driver)
except Exception as e:
print(driver.current_url, e)
closetor(driver)
new_parse(forumName, baseURL, False)
# Opens Tor Browser
def opentor():
from Forums.Initialization.forums_mining import config
global pid
print("Connecting Tor...")
pro = subprocess.Popen(config.get('TOR', 'firefox_binary_path'))
pid = pro.pid
time.sleep(7.5)
input('Tor Connected. Press ENTER to continue\n')
return
# Login using premade account credentials and do login captcha manually
def login(driver):
input('Press enter when CAPTCHA is completed, and you\'re at the login page')
#entering username and password into input boxes
usernameBox = driver.find_element(by=By.NAME, value='login')
#Username here
usernameBox.send_keys('ct1234')#sends string to the username box
passwordBox = driver.find_element(by=By.NAME, value='password')
#Password here
passwordBox.send_keys('r5o0wqmw')# sends string to passwordBox
login = driver.find_element(by=By.CLASS_NAME, value='block-container')
login_link = login.find_element(by=By.TAG_NAME, value='button')
login_link.click()
# input('input')
# wait for listing page show up (This Xpath may need to change based on different seed url)
# wait for 50 sec until id = tab_content is found, then cont
WebDriverWait(driver, 50).until(EC.visibility_of_element_located(
(By.TAG_NAME, 'nav')))
# click link to correct forum board
login_link = driver.find_element(by=By.XPATH, value='/html/body/nav/div[1]/a[3]').get_attribute('href')
driver.get(login_link) # open tab with url
# wait for listing page show up (This Xpath may need to change based on different seed url)
# wait for 50 sec until id = tab_content is found, then cont
WebDriverWait(driver, 50).until(EC.visibility_of_element_located(
(By.XPATH, '/html/body/div/div/div[3]/div[5]')))
# Returns the name of the website
def getForumName() -> str:
name = 'Libre'
return name
# Return the link of the website
def getFixedURL():
url = 'http://libreeunomyly6ot7kspglmbd5cvlkogib6rozy43r2glatc6rmwauqd.onion/'
return url
# Closes Tor Browser
def closetor(driver):
# global pid
# os.system("taskkill /pid " + str(pro.pid))
# os.system("taskkill /t /f /im tor.exe")
print('Closing Tor...')
driver.close() #close tab
time.sleep(3)
return
# Creates FireFox 'driver' and configure its 'Profile'
# to use Tor proxy and socket
def createFFDriver():
from Forums.Initialization.forums_mining import config
ff_binary = FirefoxBinary(config.get('TOR', 'firefox_binary_path'))
ff_prof = FirefoxProfile(config.get('TOR', 'firefox_profile_path'))
ff_prof.set_preference("places.history.enabled", False)
ff_prof.set_preference("privacy.clearOnShutdown.offlineApps", True)
ff_prof.set_preference("privacy.clearOnShutdown.passwords", True)
ff_prof.set_preference("privacy.clearOnShutdown.siteSettings", True)
ff_prof.set_preference("privacy.sanitize.sanitizeOnShutdown", True)
ff_prof.set_preference("signon.rememberSignons", False)
ff_prof.set_preference("network.cookie.lifetimePolicy", 2)
ff_prof.set_preference("network.dns.disablePrefetch", True)
ff_prof.set_preference("network.http.sendRefererHeader", 0)
ff_prof.set_preference("permissions.default.image", 3)
ff_prof.set_preference("browser.download.folderList", 2)
ff_prof.set_preference("browser.download.manager.showWhenStarting", False)
ff_prof.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/plain")
ff_prof.set_preference('network.proxy.type', 1)
ff_prof.set_preference("network.proxy.socks_version", 5)
ff_prof.set_preference('network.proxy.socks', '127.0.0.1')
ff_prof.set_preference('network.proxy.socks_port', 9150)
ff_prof.set_preference('network.proxy.socks_remote_dns', True)
ff_prof.set_preference("javascript.enabled", True)
ff_prof.update_preferences()
service = Service(config.get('TOR', 'geckodriver_path'))
driver = webdriver.Firefox(firefox_binary=ff_binary, firefox_profile=ff_prof, service=service)
return driver
def getAccess():
url = getFixedURL()
driver = createFFDriver()
try:
driver.get(url)
return driver
except:
driver.close()
return 'down'
# Saves the crawled html page
def savePage(page, url):
cleanPage = cleanHTML(page)
filePath = getFullPathName(url)
os.makedirs(os.path.dirname(filePath), exist_ok=True)
open(filePath, 'wb').write(cleanPage.encode('utf-8'))
return
# Gets the full path of the page to be saved along with its appropriate file name
def getFullPathName(url):
from Forums.Initialization.forums_mining import config, CURRENT_DATE
mainDir = os.path.join(config.get('Project', 'shared_folder'), "Forums/" + getForumName() + "/HTML_Pages")
fileName = getNameFromURL(url)
if isDescriptionLink(url):
fullPath = os.path.join(mainDir, CURRENT_DATE + r'\\Description\\' + fileName + '.html')
else:
fullPath = os.path.join(mainDir, CURRENT_DATE + r'\\Listing\\' + fileName + '.html')
return fullPath
# Creates the file name from passed URL
def getNameFromURL(url):
global counter
name = ''.join(e for e in url if e.isalnum())
if name == '':
name = str(counter)
counter = counter + 1
return name
def getInterestedLinks():
links = []
# # cyber security
links.append('http://libreeunomyly6ot7kspglmbd5cvlkogib6rozy43r2glatc6rmwauqd.onion/c/CyberSecurity')
# # services
# links.append('http://libreeunomyly6ot7kspglmbd5cvlkogib6rozy43r2glatc6rmwauqd.onion/c/Services')
# # programming
# links.append('http://libreeunomyly6ot7kspglmbd5cvlkogib6rozy43r2glatc6rmwauqd.onion/c/Programming')
return links
def crawlForum(driver):
print("Crawling the Libre forum")
linksToCrawl = getInterestedLinks()
visited = set(linksToCrawl)
initialTime = time.time()
i = 0
count = 0
while i < len(linksToCrawl):
link = linksToCrawl[i]
print('Crawling :', link)
try:
try:
driver.get(link)
except:
driver.refresh()
html = driver.page_source
savePage(html, link)
has_next_page = True
while has_next_page:
list = topicPages(html)
for item in list:
itemURL = urlparse.urljoin(baseURL, str(item))
try:
driver.get(itemURL)
except:
driver.refresh()
savePage(driver.page_source, item)
driver.back()
#variable to check if there is a next page for the topic
# has_next_topic_page = True
# counter = 1
# # check if there is a next page for the topics
# while has_next_topic_page:
# # try to access next page of th topic
# itemURL = urlparse.urljoin(baseURL, str(item))
# try:
# driver.get(itemURL)
# except:
# driver.refresh()
# savePage(driver.page_source, item)
#
# # if there is a next page then go and save....
# # Spec
# try:
# # temp = driver.find_element(By.XPATH, '/html/body/div[2]/div[4]/div/div[5]/div[2]/div/div[1]/div[1]/div/nav/div[1]') # /html/body/div/div[2]/div/div[2]/div/
# item = driver.find_element(by=By.LINK_TEXT, value='>').get_attribute('href') #/html/body/div/div[2]/div/div[2]/div
#
# if item == "":
# raise NoSuchElementException
# else:
# counter += 1
#
# except NoSuchElementException:
# has_next_topic_page = False
#
# # end of loop
# for i in range(counter):
# driver.back()
# comment out
break
# comment out
if count == 1:
count = 0
break
try:
# temp = driver.find_element(by=By.XPATH, value = '/html/body/div[2]/div[4]/div/div[5]/div[2]/div/div/div[1]/div/nav/div[1]')
link = driver.find_element(by=By.LINK_TEXT, value='>').get_attribute('href')
if link == "":
raise NoSuchElementException
try:
driver.get(link)
except:
driver.refresh()
html = driver.page_source
savePage(html, link)
count += 1
except NoSuchElementException:
has_next_page = False
except Exception as e:
print(link, e)
i += 1
# finalTime = time.time()
# print finalTime - initialTime
input("Crawling Libre forum done successfully. Press ENTER to continue\n")
# Returns 'True' if the link is Topic link, may need to change for every website
def isDescriptionLink(url):
if '/p/' in url:
return True
return False
# Returns True if the link is a listingPage link, may need to change for every website
def isListingLink(url):
if '/c/' in url:
return True
return False
# calling the parser to define the links
def topicPages(html):
soup = BeautifulSoup(html, "html.parser")
return libre_links_parser(soup)
def crawler():
startCrawling()
# print("Crawling and Parsing BestCardingWorld .... DONE!")

+ 247
- 0
Forums/Libre/parser.py View File

@ -0,0 +1,247 @@
__author__ = 'DarkWeb'
# Here, we are importing the auxiliary functions to clean or convert data
from Forums.Utilities.utilities import *
from datetime import date
from datetime import timedelta
import re
# Here, we are importing BeautifulSoup to search through the HTML tree
from bs4 import BeautifulSoup
# This is the method to parse the Description Pages (one page to each topic in the Listing Pages)
def cryptBB_description_parser(soup):
# Fields to be parsed
topic = "-1" # 0 *topic name
user = [] # 1 *all users of each post
status = [] # 2 all user's authority in each post such as (adm, member, dangerous)
reputation = [] # 3 all user's karma in each post (usually found as a number)
interest = [] # 4 all user's interest in each post
sign = [] # 5 all user's signature in each post (usually a standard message after the content of the post)
post = [] # 6 all messages of each post
feedback = [] # 7 all feedbacks of each vendor (this was found in just one Forum and with a number format)
addDate = [] # 8 all dates of each post
# Finding the topic (should be just one coming from the Listing Page)
li = soup.find("td", {"class": "thead"}).find('strong')
topic = li.text
topic = re.sub("\[\w*\]", '', topic)
topic = topic.replace(",","")
topic = topic.replace("\n","")
topic = cleanString(topic.strip())
# Finding the repeated tag that corresponds to the listing of posts
# try:
posts = soup.find('table', {"class": "tborder tfixed clear"}).find('td', {"id": "posts_container"}).find_all(
'div', {"class": "post"})
# For each message (post), get all the fields we are interested to:
for ipost in posts:
# Finding a first level of the HTML page
post_wrapper = ipost.find('span', {"class": "largetext"})
# Finding the author (user) of the post
author = post_wrapper.text.strip()
user.append(cleanString(author)) # Remember to clean the problematic characters
# Finding the status of the author
smalltext = ipost.find('div', {"class": "post_author"})
'''
# Testing here two possibilities to find this status and combine them
if ipost.find('div', {"class": "deleted_post_author"}):
status.append(-1)
interest.append(-1)
reputation.append(-1)
addDate.append(-1)
post.append("THIS POST HAS BEEN REMOVED!")
sign.append(-1)
feedback.append(-1)
continue
'''
# CryptBB does have membergroup and postgroup
membergroup = smalltext.find('div', {"class": "profile-rank"})
postgroup = smalltext.find('div', {"class": "postgroup"})
if membergroup != None:
membergroup = membergroup.text.strip()
if postgroup != None:
postgroup = postgroup.text.strip()
membergroup = membergroup + " - " + postgroup
else:
if postgroup != None:
membergroup = postgroup.text.strip()
else:
membergroup = "-1"
status.append(cleanString(membergroup))
# Finding the interest of the author
# CryptBB does not have blurb
blurb = smalltext.find('li', {"class": "blurb"})
if blurb != None:
blurb = blurb.text.strip()
else:
blurb = "-1"
interest.append(cleanString(blurb))
# Finding the reputation of the user
# CryptBB does have reputation
author_stats = smalltext.find('div', {"class": "author_statistics"})
karma = author_stats.find('strong')
if karma != None:
karma = karma.text
karma = karma.replace("Community Rating: ", "")
karma = karma.replace("Karma: ", "")
karma = karma.strip()
else:
karma = "-1"
reputation.append(cleanString(karma))
# Getting here another good tag to find the post date, post content and users' signature
postarea = ipost.find('div', {"class": "post_content"})
dt = postarea.find('span', {"class": "post_date"}).text
# dt = dt.strip().split()
dt = dt.strip()
day=date.today()
if "Yesterday" in dt:
yesterday = day - timedelta(days=1)
yesterday = yesterday.strftime('%m-%d-%Y')
stime = dt.replace('Yesterday,','').strip()
date_time_obj = yesterday+ ', '+stime
date_time_obj = datetime.strptime(date_time_obj,'%m-%d-%Y, %I:%M %p')
elif "hours ago" in dt:
day = day.strftime('%m-%d-%Y')
date_time_obj = postarea.find('span', {"class": "post_date"}).find('span')['title']
date_time_obj = datetime.strptime(date_time_obj, '%m-%d-%Y, %I:%M %p')
else:
date_time_obj = datetime.strptime(dt, '%m-%d-%Y, %I:%M %p')
stime = date_time_obj.strftime('%b %d, %Y')
sdate = date_time_obj.strftime('%I:%M %p')
addDate.append(date_time_obj)
# Finding the post
inner = postarea.find('div', {"class": "post_body scaleimages"})
inner = inner.text.strip()
post.append(cleanString(inner))
# Finding the user's signature
# signature = ipost.find('div', {"class": "post_wrapper"}).find('div', {"class": "moderatorbar"}).find('div', {"class": "signature"})
signature = ipost.find('div', {"class": "signature scaleimages"})
if signature != None:
signature = signature.text.strip()
# print(signature)
else:
signature = "-1"
sign.append(cleanString(signature))
# As no information about user's feedback was found, just assign "-1" to the variable
feedback.append("-1")
# Populate the final variable (this should be a list with all fields scraped)
row = (topic, user, status, reputation, interest, sign, post, feedback, addDate)
# Sending the results
return row
# This is the method to parse the Listing Pages (one page with many posts)
def cryptBB_listing_parser(soup):
nm = 0 # *this variable should receive the number of topics
forum = "OnniForums" # 0 *forum name
board = "-1" # 1 *board name (the previous level of the topic in the Forum categorization tree.
# For instance: Security/Malware/Tools to hack Facebook. The board here should be Malware)
author = [] # 2 *all authors of each topic
topic = [] # 3 *all topics
views = [] # 4 number of views of each topic
posts = [] # 5 number of posts of each topic
href = [] # 6 this variable should receive all cleaned urls (we will use this to do the marge between
# Listing and Description pages)
addDate = [] # 7 when the topic was created (difficult to find)
# Finding the board (should be just one)
board = soup.find('span', {"class": "active"}).text
board = cleanString(board.strip())
# Finding the repeated tag that corresponds to the listing of topics
itopics = soup.find_all('tr',{"class": "inline_row"})
for itopic in itopics:
# For each topic found, the structure to get the rest of the information can be of two types. Testing all of them
# to don't miss any topic
# Adding the topic to the topic list
try:
topics = itopic.find('span', {"class": "subject_old"}).find('a').text
except:
topics = itopic.find('span', {"class": "subject_new"}).find('a').text
topics = re.sub("\[\w*\]", '', topics)
topic.append(cleanString(topics))
# Counting how many topics we have found so far
nm = len(topic)
# Adding the url to the list of urls
try:
link = itopic.find('span', {"class": "subject_old"}).find('a').get('href')
except:
link = itopic.find('span',{"class": "subject_new"}).find('a').get('href')
href.append(link)
# Finding the author of the topic
ps = itopic.find('div', {"class":"author smalltext"}).find('a').text
user = ps.strip()
author.append(cleanString(user))
# Finding the number of replies
columns = itopic.findChildren('td',recursive=False)
replies = columns[3].text
posts.append(cleanString(replies))
# Finding the number of Views
tview = columns[4].text
views.append(cleanString(tview))
# If no information about when the topic was added, just assign "-1" to the variable
addDate.append("-1")
return organizeTopics(forum, nm, board, author, topic, views, posts, href, addDate)
def libre_links_parser(soup):
# Returning all links that should be visited by the Crawler
href = []
listing = soup.find_all('div', {"class": "flex-grow p-2 text-justify"})
for a in listing:
link = a.find('div', {'class': 'flex space-x-2 items-center'}).find('a').get('href')
href.append(link)
return href

BIN
Forums/OnniForums/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
Forums/OnniForums/__pycache__/crawler_selenium.cpython-311.pyc View File


BIN
Forums/OnniForums/__pycache__/parser.cpython-310.pyc View File


BIN
Forums/OnniForums/__pycache__/parser.cpython-311.pyc View File


BIN
Forums/OnniForums/__pycache__/parser_script.cpython-311.pyc View File


+ 16
- 16
Forums/OnniForums/crawler_selenium.py View File

@ -31,19 +31,19 @@ baseURL = 'http://onnii6niq53gv3rvjpi7z5axkasurk2x5w5lwliep4qyeb2azagxn4qd.onion
# Opens Tor Browser, crawls the website # Opens Tor Browser, crawls the website
def startCrawling(): def startCrawling():
# opentor()
opentor()
forumName = getForumName() forumName = getForumName()
# driver = getAccess()
driver = getAccess()
# if driver != 'down':
# try:
# login(driver)
# crawlForum(driver)
# except Exception as e:
# print(driver.current_url, e)
# closetor(driver)
if driver != 'down':
try:
login(driver)
crawlForum(driver)
except Exception as e:
print(driver.current_url, e)
closetor(driver)
new_parse(forum=forumName, url=baseURL, createLog=False)
# new_parse(forum=forumName, url=baseURL, createLog=False)
# Opens Tor Browser # Opens Tor Browser
@ -190,9 +190,9 @@ def getInterestedLinks():
links = [] links = []
# # Hacking & Cracking tutorials # # Hacking & Cracking tutorials
# links.append('http://onnii6niq53gv3rvjpi7z5axkasurk2x5w5lwliep4qyeb2azagxn4qd.onion/Forum-Hacking-Cracking-tutorials')
links.append('http://onnii6niq53gv3rvjpi7z5axkasurk2x5w5lwliep4qyeb2azagxn4qd.onion/Forum-Hacking-Cracking-tutorials')
# Hacking & Cracking questions # Hacking & Cracking questions
links.append('http://onnii6niq53gv3rvjpi7z5axkasurk2x5w5lwliep4qyeb2azagxn4qd.onion/Forum-Hacking-Cracking-questions')
# links.append('http://onnii6niq53gv3rvjpi7z5axkasurk2x5w5lwliep4qyeb2azagxn4qd.onion/Forum-Hacking-Cracking-questions')
# # Exploit PoCs # # Exploit PoCs
# links.append('http://onnii6niq53gv3rvjpi7z5axkasurk2x5w5lwliep4qyeb2azagxn4qd.onion/Forum-Exploit-PoCs') # links.append('http://onnii6niq53gv3rvjpi7z5axkasurk2x5w5lwliep4qyeb2azagxn4qd.onion/Forum-Exploit-PoCs')
# # Cracked software # # Cracked software
@ -280,12 +280,12 @@ def crawlForum(driver):
driver.back() driver.back()
# comment out, one topic per page # comment out, one topic per page
# break
break
# comment out, go through all pages # comment out, go through all pages
# if count == 1:
# count = 0
# break
if count == 1:
count = 0
break
try: try:
temp = driver.find_element(by=By.XPATH, value= temp = driver.find_element(by=By.XPATH, value=


+ 462
- 0
Forums/OnniForums/geckodriver.log View File

@ -0,0 +1,462 @@
1655762162153 geckodriver INFO Listening on 127.0.0.1:50333
1655762166434 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50334" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqUeAAN"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762166904 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50334/devtools/browser/fb880205-35da-44a4-83b8-a861ce7125f1
1655762168590 Marionette INFO Listening on port 50341
1655762169104 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655762207624 Marionette INFO Stopped listening on port 50341
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655762207923 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655762990320 geckodriver INFO Listening on 127.0.0.1:50884
1655762994595 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50885" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileurjxEe"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655762995244 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50885/devtools/browser/36503f58-7ab7-4188-81b0-fa1b81f32c0a
1655762996997 Marionette INFO Listening on port 50890
JavaScript error: resource://gre/modules/ExtensionContent.jsm, line 575: TypeError: PrecompiledScript.executeInGlobal: Argument 1 is not an object.
1655762997277 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763083091 Marionette INFO Stopped listening on port 50890
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763083216 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763089557 geckodriver INFO Listening on 127.0.0.1:50923
1655763093759 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50924" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile5mY1qq"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763094208 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50924/devtools/browser/68ce2df9-521f-4258-94f7-c2e2b199701a
1655763095918 Marionette INFO Listening on port 50929
1655763096408 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763143935 Marionette INFO Stopped listening on port 50929
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655763144495 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763166975 geckodriver INFO Listening on 127.0.0.1:50947
1655763171175 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50948" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilenAGZM5"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763171719 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50948/devtools/browser/0b9966bd-0cec-45cc-9d11-02a2580233f0
1655763173307 Marionette INFO Listening on port 50953
1655763173368 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763208922 Marionette INFO Stopped listening on port 50953
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763209261 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655763387409 geckodriver INFO Listening on 127.0.0.1:49752
1655763391806 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "49753" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilevAHJSJ"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655763392827 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:49753/devtools/browser/7fe49a16-940f-4ee8-9a2f-48739a92db78
1655763394700 Marionette INFO Listening on port 49759
1655763395103 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655763430848 Marionette INFO Stopped listening on port 49759
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655763431145 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655765589286 geckodriver INFO Listening on 127.0.0.1:50102
1655765621677 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50103" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ecUpb"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655765622149 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50103/devtools/browser/e5a916fc-987c-43d4-9c12-9c9b88cca242
1655765623776 Marionette INFO Listening on port 50109
1655765623892 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766125168 Marionette INFO Stopped listening on port 50109
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655766125303 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655766505099 geckodriver INFO Listening on 127.0.0.1:50156
1655766509351 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50157" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileVqvmzh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655766509876 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50157/devtools/browser/13084c14-64d3-48a5-8b99-4c514e961d4f
1655766511328 Marionette INFO Listening on port 50162
1655766511395 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655766552298 Marionette INFO Stopped listening on port 50162
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
JavaScript error: chrome://remote/content/marionette/cert.js, line 55: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsICertOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData]
!!! error running onStopped callback: TypeError: callback is not a function
1655766552578 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767657126 geckodriver INFO Listening on 127.0.0.1:50257
1655767661375 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50258" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileB9Dzeh"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767661843 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50258/devtools/browser/867cd318-6430-4b3f-9b63-8c6bdf17636a
1655767663330 Marionette INFO Listening on port 50263
1655767663431 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767668263 Marionette INFO Stopped listening on port 50263
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655767668591 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767689480 geckodriver INFO Listening on 127.0.0.1:50276
1655767693635 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50277" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile8ZOQe7"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767694215 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50277/devtools/browser/a70a3f5b-19af-4a68-99c8-c46086ba9599
1655767695725 Marionette INFO Listening on port 50282
1655767695830 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767701272 Marionette INFO Stopped listening on port 50282
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767701594 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767791360 geckodriver INFO Listening on 127.0.0.1:50296
1655767795575 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50297" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile4GEqBr"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767796001 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50297/devtools/browser/e2d4e57a-16ce-429d-9804-e9e391199ddf
1655767797534 Marionette INFO Listening on port 50302
1655767797661 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767802833 Marionette INFO Stopped listening on port 50302
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767803151 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767902908 geckodriver INFO Listening on 127.0.0.1:50320
1655767907009 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50321" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileqlJ8ZA"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767907484 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
DevTools listening on ws://localhost:50321/devtools/browser/49e77669-01dd-447e-a804-36d42b3400cf
1655767908782 Marionette INFO Listening on port 50326
1655767909069 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767916218 Marionette INFO Stopped listening on port 50326
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767916523 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767958348 geckodriver INFO Listening on 127.0.0.1:50340
1655767962557 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50341" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofilekpCdfs"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655767963032 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50341/devtools/browser/b4d1728c-d26f-4537-adc2-5c954b171b13
1655767964591 Marionette INFO Listening on port 50346
1655767964633 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655767973210 Marionette INFO Stopped listening on port 50346
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655767973501 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655767998084 geckodriver INFO Listening on 127.0.0.1:50358
1655768002314 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50359" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofileujYNBj"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768002757 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50359/devtools/browser/04112c45-204d-43f1-9615-34782fd06632
1655768004165 Marionette INFO Listening on port 50364
1655768004375 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768009787 Marionette INFO Stopped listening on port 50364
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
1655768010100 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768083966 geckodriver INFO Listening on 127.0.0.1:50386
1655768088234 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50387" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile39Pl1f"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768088664 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50387/devtools/browser/3d6cda42-cb60-4c67-8d76-1d186a51887f
1655768090263 Marionette INFO Listening on port 50392
1655768090299 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768091655 Marionette INFO Stopped listening on port 50392
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x390076,name=PContent::Msg_DestroyBrowsingContextGroup) Closed channel: cannot send/recv
1655768092659 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768101201 geckodriver INFO Listening on 127.0.0.1:50404
1655768105373 mozrunner::runner INFO Running command: "C:\\Users\\CALSysLab\\Desktop\\Tor Browser\\Browser\\firefox.exe" "--marionette" "--remote-debugging-port" "50405" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\CALSYS~1\\AppData\\Local\\Temp\\rust_mozprofile2YRGSz"
console.log: "TorSettings: loadFromPrefs()"
console.log: "TorConnect: Init"
console.log: "TorConnect: observed profile-after-change"
console.log: "TorConnect: observing topic 'TorBootstrapStatus'"
console.log: "TorConnect: observing topic 'TorBootstrapError'"
console.log: "TorConnect: observing topic 'TorProcessExited'"
console.log: "TorConnect: observing topic 'TorLogHasWarnOrErr'"
console.log: "TorConnect: observing topic 'torsettings:ready'"
console.log: "TorSettings: observed profile-after-change"
1655768105853 Marionette INFO Marionette enabled
console.log: "TorConnect: will load after bootstrap => [about:blank]"
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
JavaScript error: resource://gre/modules/XPCOMUtils.jsm, line 161: TypeError: Cc[aContract] is undefined
console.error: "Could not load engine [email protected]: Error: Extension is invalid"
DevTools listening on ws://localhost:50405/devtools/browser/9c4a4d71-25ff-4980-a54d-8545d6200790
1655768107431 Marionette INFO Listening on port 50410
1655768107530 RemoteAgent WARN TLS certificate errors will be ignored for this session
1655768119155 Marionette INFO Stopped listening on port 50410
JavaScript error: resource:///modules/Interactions.jsm, line 209: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
!!! error running onStopped callback: TypeError: callback is not a function
1655768119438 RemoteAgent ERROR unable to stop listener: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIWindowMediator.getEnumerator]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://remote/content/cdp/observers/TargetObserver.jsm :: stop :: line 64" data: no] Stack trace: stop()@TargetObserver.jsm:64
unwatchForTabs()@TargetList.jsm:70
unwatchForTargets()@TargetList.jsm:37
destructor()@TargetList.jsm:109
stop()@CDP.jsm:104
close()@RemoteAgent.jsm:138
1655768229472 geckodriver INFO Listening on 127.0.0.1:50431
1655768233458 webdriver::server WARN Rejected request with missing Host header

+ 320
- 0
Forums/Procrax/crawler_selenium.py View File

@ -0,0 +1,320 @@
__author__ = 'Helium'
'''
Procrax Forum Crawler (Selenium)
rechecked and confirmed
'''
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from PIL import Image
import urllib.parse as urlparse
import os, re, time
from datetime import date
import configparser
import subprocess
from bs4 import BeautifulSoup
from Forums.Initialization.prepare_parser import new_parse
from Forums.Procrax.parser import procrax_links_parser
from Forums.Utilities.utilities import cleanHTML
counter = 1
baseURL = 'https://procrax.cx/'
# Opens Tor Browser, crawls the website
def startCrawling():
opentor()
# forumName = getForumName()
driver = getAccess()
if driver != 'down':
try:
login(driver)
crawlForum(driver)
except Exception as e:
print(driver.current_url, e)
closetor(driver)
# new_parse(forumName, False)
# Opens Tor Browser
def opentor():
from Forums.Initialization.forums_mining import config
global pid
print("Connecting Tor...")
pro = subprocess.Popen(config.get('TOR', 'firefox_binary_path'))
pid = pro.pid
time.sleep(7.5)
input('Tor Connected. Press ENTER to continue\n')
return
# Login using premade account credentials and do login captcha manually
def login(driver):
WebDriverWait(driver, 50).until(EC.visibility_of_element_located(
(By.XPATH, '/html/body/div[1]/div[3]/div[2]/div[3]/div[2]/div[1]/form/div/div/div/dl[4]/dd/div/div[2]/button/span')))
#entering username and password into input boxes
usernameBox = driver.find_element(by=By.NAME, value='login')
#Username here
usernameBox.send_keys('cheese_pizza_man')#sends string to the username box
passwordBox = driver.find_element(by=By.NAME, value='password')
#Password here
passwordBox.send_keys('Gr33nSp@m&3ggs')# sends string to passwordBox
clicker = driver.find_element(by=By.XPATH, value='/html/body/div[1]/div[3]/div[2]/div[3]/div[2]/div[1]/form/div/div/div/dl[4]/dd/div/div[2]/button/span')
clicker.click()
# # wait for listing page show up (This Xpath may need to change based on different seed url)
# # wait for 50 sec until id = tab_content is found, then cont
WebDriverWait(driver, 50).until(EC.visibility_of_element_located(
(By.XPATH, '/html/body/div[1]/div[3]/div[2]/div[3]/div[1]/div/div[1]/div')))
# Returns the name of the website
def getForumName():
name = 'Procrax'
return name
# Return the link of the website
def getFixedURL():
url = 'https://procrax.cx/'
return url
# Closes Tor Browser
def closetor(driver):
# global pid
# os.system("taskkill /pid " + str(pro.pid))
# os.system("taskkill /t /f /im tor.exe")
print('Closing Tor...')
driver.close() #close tab
time.sleep(3)
return
# Creates FireFox 'driver' and configure its 'Profile'
# to use Tor proxy and socket
def createFFDriver():
from Forums.Initialization.forums_mining import config
ff_binary = FirefoxBinary(config.get('TOR', 'firefox_binary_path'))
ff_prof = FirefoxProfile(config.get('TOR', 'firefox_profile_path'))
ff_prof.set_preference("places.history.enabled", False)
ff_prof.set_preference("privacy.clearOnShutdown.offlineApps", True)
ff_prof.set_preference("privacy.clearOnShutdown.passwords", True)
ff_prof.set_preference("privacy.clearOnShutdown.siteSettings", True)
ff_prof.set_preference("privacy.sanitize.sanitizeOnShutdown", True)
ff_prof.set_preference("signon.rememberSignons", False)
ff_prof.set_preference("network.cookie.lifetimePolicy", 2)
ff_prof.set_preference("network.dns.disablePrefetch", True)
ff_prof.set_preference("network.http.sendRefererHeader", 0)
ff_prof.set_preference("permissions.default.image", 3)
ff_prof.set_preference("browser.download.folderList", 2)
ff_prof.set_preference("browser.download.manager.showWhenStarting", False)
ff_prof.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/plain")
ff_prof.set_preference('network.proxy.type', 1)
ff_prof.set_preference("network.proxy.socks_version", 5)
ff_prof.set_preference('network.proxy.socks', '127.0.0.1')
ff_prof.set_preference('network.proxy.socks_port', 9150)
ff_prof.set_preference('network.proxy.socks_remote_dns', True)
ff_prof.set_preference("javascript.enabled", True)
ff_prof.update_preferences()
service = Service(config.get('TOR', 'geckodriver_path'))
driver = webdriver.Firefox(firefox_binary=ff_binary, firefox_profile=ff_prof, service=service)
return driver
def getAccess():
url = getFixedURL()
driver = createFFDriver()
try:
driver.get(url)# open url in browser
return driver
except:
driver.close()# close tab
return 'down'
# Saves the crawled html page
def savePage(page, url):
cleanPage = cleanHTML(page)
filePath = getFullPathName(url)
os.makedirs(os.path.dirname(filePath), exist_ok=True)
open(filePath, 'wb').write(cleanPage.encode('utf-8'))
return
# Gets the full path of the page to be saved along with its appropriate file name
def getFullPathName(url):
from Forums.Initialization.forums_mining import config, CURRENT_DATE
mainDir = os.path.join(config.get('Project', 'shared_folder'), "Forums/" + getForumName() + "/HTML_Pages")
fileName = getNameFromURL(url)
if isDescriptionLink(url):
fullPath = os.path.join(mainDir, CURRENT_DATE + r'\\Description\\' + fileName + '.html')
else:
fullPath = os.path.join(mainDir, CURRENT_DATE + r'\\Listing\\' + fileName + '.html')
return fullPath
# Creates the file name from passed URL
def getNameFromURL(url):
global counter
name = ''.join(e for e in url if e.isalnum())
if (name == ''):
name = str(counter)
counter = counter + 1
return name
def getInterestedLinks():
links = []
# # general hacking
# links.append('https://procrax.cx/forums/general-hacking.24/')
# # hacking security tools
# links.append('https://procrax.cx/forums/hacking-security-tools.20/')
# # hacktube
# links.append('https://procrax.cx/forums/hacktube.22/')
# # cardable
# links.append('https://procrax.cx/forums/cardable-websites.28/')
# # tools
# links.append('https://procrax.cx/forums/tools-bots-validators.73/')
# general forum
links.append('https://procrax.cx/forums/forum-discussions-updates.7/')
return links
def crawlForum(driver):
print("Crawling the Procrax forum")
linksToCrawl = getInterestedLinks()
visited = set(linksToCrawl)
initialTime = time.time()
i = 0
count = 0
while i < len(linksToCrawl):
link = linksToCrawl[i]
print('Crawling :', link)
try:
try:
driver.get(link)# open
except:
driver.refresh()
html = driver.page_source
savePage(html, link)
has_next_page = True
#loop through the topics
while has_next_page:
list = topicPages(html)# for multiple pages
for item in list:
#variable to check if there is a next page for the topic
has_next_topic_page = True
counter = 1
# check if there is a next page for the topics
while has_next_topic_page:
# try to access next page of th topic
itemURL = urlparse.urljoin(baseURL, str(item))
try:
driver.get(itemURL)
except:
driver.refresh()
savePage(driver.page_source, item)
# if there is a next page then go and save....
# specific
try:
# temp = driver.find_element(By.XPATH, value='/html/body/div[1]/div[3]/div[2]/div[3]/div/div')
item = driver.find_element(By.LINK_TEXT, value='Next').get_attribute('href')
if item == "":
raise NoSuchElementException
has_next_topic_page = False
else:
counter += 1
except NoSuchElementException:
has_next_topic_page = False
#end of loop
for i in range(counter):
driver.back()
# # comment out
# break
#
# # comment out
# if count == 1:
# count = 0
# break
try:# change depending on web page, #general
# /html/body/div[1]/div[3]/div[2]/div[3]/div/div/div/div[1]/div/nav/div[1]
# temp = driver.find_element(By.XPATH, value='/html/body/div[1]/div[3]/div[2]/div[3]/div/div/div/div[1]/div/nav/div[1]')
link = driver.find_element(by=By.LINK_TEXT, value='Next').get_attribute('href')
if link == "":
raise NoSuchElementException
try:
driver.get(link)
except:
driver.refresh()
html = driver.page_source
savePage(html, link)
count += 1
except NoSuchElementException:
has_next_page = False
except Exception as e:
print(link, e)
i += 1
# finalTime = time.time()
# print finalTime - initialTime
input("Crawling Procrax forum done successfully. Press ENTER to continue\n")
# Returns 'True' if the link is Topic link, may need to change for every website
def isDescriptionLink(url):
if 'threads' in url:
return True
return False
# Returns True if the link is a listingPage link, may need to change for every website
def isListingLink(url):
if 'forums' in url:
return True
return False
# calling the parser to define the links
def topicPages(html):
soup = BeautifulSoup(html, "html.parser")
#print(soup.find('div', id="container").find('div', id="content").find('table', {"class": "tborder clear"}).find('tbody').find('tr',{"class": "inline_row"}).find('strong').text)
return procrax_links_parser(soup)
def crawler():
startCrawling()
# print("Crawling and Parsing BestCardingWorld .... DONE!")

+ 264
- 0
Forums/Procrax/parser.py View File

@ -0,0 +1,264 @@
__author__ = 'Helium'
# Here, we are importing the auxiliary functions to clean or convert data
from Forums.Utilities.utilities import *
from datetime import date
from datetime import timedelta
import re
# Here, we are importing BeautifulSoup to search through the HTML tree
from bs4 import BeautifulSoup
# This is the method to parse the Description Pages (one page to each topic in the Listing Pages)
def cryptBB_description_parser(soup):
# Fields to be parsed
topic = "-1" # topic name
user = [] # all users of each post
addDate = [] # all dated of each post
feedback = [] # all feedbacks of each vendor (this was found in just one Forum and with a number format)
status = [] # all user's authority in each post such as (adm, member, dangerous)
reputation = [] # all user's karma in each post (usually found as a number)
sign = [] # all user's signature in each post (usually a standard message after the content of the post)
post = [] # all messages of each post
interest = [] # all user's interest in each post
# Finding the topic (should be just one coming from the Listing Page)
li = soup.find("td", {"class": "thead"}).find('strong')
topic = li.text
topic = re.sub("\[\w*\]", '', topic)
topic = topic.replace(",","")
topic = topic.replace("\n","")
topic = cleanString(topic.strip())
# Finding the repeated tag that corresponds to the listing of posts
# try:
posts = soup.find('table', {"class": "tborder tfixed clear"}).find('td', {"id": "posts_container"}).find_all(
'div', {"class": "post"})
# For each message (post), get all the fields we are interested to:
for ipost in posts:
# Finding a first level of the HTML page
post_wrapper = ipost.find('span', {"class": "largetext"})
# Finding the author (user) of the post
author = post_wrapper.text.strip()
user.append(cleanString(author)) # Remember to clean the problematic characters
# Finding the status of the author
smalltext = ipost.find('div', {"class": "post_author"})
'''
# Testing here two possibilities to find this status and combine them
if ipost.find('div', {"class": "deleted_post_author"}):
status.append(-1)
interest.append(-1)
reputation.append(-1)
addDate.append(-1)
post.append("THIS POST HAS BEEN REMOVED!")
sign.append(-1)
feedback.append(-1)
continue
'''
# CryptBB does have membergroup and postgroup
membergroup = smalltext.find('div', {"class": "profile-rank"})
postgroup = smalltext.find('div', {"class": "postgroup"})
if membergroup != None:
membergroup = membergroup.text.strip()
if postgroup != None:
postgroup = postgroup.text.strip()
membergroup = membergroup + " - " + postgroup
else:
if postgroup != None:
membergroup = postgroup.text.strip()
else:
membergroup = "-1"
status.append(cleanString(membergroup))
# Finding the interest of the author
# CryptBB does not have blurb
blurb = smalltext.find('li', {"class": "blurb"})
if blurb != None:
blurb = blurb.text.strip()
else:
blurb = "-1"
interest.append(cleanString(blurb))
# Finding the reputation of the user
# CryptBB does have reputation
author_stats = smalltext.find('div', {"class": "author_statistics"})
karma = author_stats.find('strong')
if karma != None:
karma = karma.text
karma = karma.replace("Community Rating: ", "")
karma = karma.replace("Karma: ", "")
karma = karma.strip()
else:
karma = "-1"
reputation.append(cleanString(karma))
# Getting here another good tag to find the post date, post content and users' signature
postarea = ipost.find('div', {"class": "post_content"})
dt = postarea.find('span', {"class": "post_date"}).text
# dt = dt.strip().split()
dt = dt.strip()
day=date.today()
if "Yesterday" in dt:
yesterday = day - timedelta(days=1)
yesterday = yesterday.strftime('%m-%d-%Y')
stime = dt.replace('Yesterday,','').strip()
date_time_obj = yesterday+ ', '+stime
date_time_obj = datetime.strptime(date_time_obj,'%m-%d-%Y, %I:%M %p')
elif "hours ago" in dt:
day = day.strftime('%m-%d-%Y')
date_time_obj = postarea.find('span', {"class": "post_date"}).find('span')['title']
date_time_obj = datetime.strptime(date_time_obj, '%m-%d-%Y, %I:%M %p')
else:
date_time_obj = datetime.strptime(dt, '%m-%d-%Y, %I:%M %p')
stime = date_time_obj.strftime('%b %d, %Y')
sdate = date_time_obj.strftime('%I:%M %p')
addDate.append(date_time_obj)
# Finding the post
inner = postarea.find('div', {"class": "post_body scaleimages"})
inner = inner.text.strip()
post.append(cleanString(inner))
# Finding the user's signature
# signature = ipost.find('div', {"class": "post_wrapper"}).find('div', {"class": "moderatorbar"}).find('div', {"class": "signature"})
signature = ipost.find('div', {"class": "signature scaleimages"})
if signature != None:
signature = signature.text.strip()
# print(signature)
else:
signature = "-1"
sign.append(cleanString(signature))
# As no information about user's feedback was found, just assign "-1" to the variable
feedback.append("-1")
'''
except:
if soup.find('td', {"class": "trow1"}).text == " You do not have permission to access this page. ":
user.append("-1")
status.append(-1)
interest.append(-1)
reputation.append(-1)
addDate.append(-1)
post.append("NO ACCESS TO THIS PAGE!")
sign.append(-1)
feedback.append(-1)
'''
# Populate the final variable (this should be a list with all fields scraped)
row = (topic, user, status, reputation, interest, sign, post, feedback, addDate)
# Sending the results
return row
# This is the method to parse the Listing Pages (one page with many posts)
def cryptBB_listing_parser(soup):
board = "-1" # board name (the previous level of the topic in the Forum categorization tree.
# For instance: Security/Malware/Tools to hack Facebook. The board here should be Malware)
nm = 0 # this variable should receive the number of topics
topic = [] # all topics
author = [] # all authors of each topic
views = [] # number of views of each topic
posts = [] # number of posts of each topic
addDate = [] # when the topic was created (difficult to find)
href = [] # this variable should receive all cleaned urls (we will use this to do the marge between
# Listing and Description pages)
# Finding the board (should be just one)
board = soup.find('span', {"class": "active"}).text
board = cleanString(board.strip())
# Finding the repeated tag that corresponds to the listing of topics
itopics = soup.find_all('tr',{"class": "inline_row"})
for itopic in itopics:
# For each topic found, the structure to get the rest of the information can be of two types. Testing all of them
# to don't miss any topic
# Adding the topic to the topic list
try:
topics = itopic.find('span', {"class": "subject_old"}).find('a').text
except:
topics = itopic.find('span', {"class": "subject_new"}).find('a').text
topics = re.sub("\[\w*\]", '', topics)
topic.append(cleanString(topics))
# Counting how many topics we have found so far
nm = len(topic)
# Adding the url to the list of urls
try:
link = itopic.find('span', {"class": "subject_old"}).find('a').get('href')
except:
link = itopic.find('span',{"class": "subject_new"}).find('a').get('href')
link = cleanLink(link)
href.append(link)
# Finding the author of the topic
ps = itopic.find('div', {"class":"author smalltext"}).find('a').text
user = ps.strip()
author.append(cleanString(user))
# Finding the number of replies
columns = itopic.findChildren('td',recursive=False)
replies = columns[3].text
posts.append(cleanString(replies))
# Finding the number of Views
tview = columns[4].text
views.append(cleanString(tview))
# If no information about when the topic was added, just assign "-1" to the variable
addDate.append("-1")
return organizeTopics("CryptBB", nm, topic, board, author, views, posts, href, addDate)
def procrax_links_parser(soup):
# Returning all links that should be visited by the Crawler
href = []
listing = soup.find_all('div', {"class": "structItem-title"})
for a in listing:
link = a.find('a', {'class': ''}).get('href')
href.append(link)
return href

BIN
Forums/Utilities/__pycache__/__init__.cpython-310.pyc View File


BIN
Forums/Utilities/__pycache__/__init__.cpython-311.pyc View File


BIN
Forums/Utilities/__pycache__/utilities.cpython-310.pyc View File


BIN
Forums/Utilities/__pycache__/utilities.cpython-311.pyc View File


BIN
Forums/__pycache__/__init__.cpython-310.pyc View File


BIN
MarketPlaces/AnonymousMarketplace/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
MarketPlaces/AnonymousMarketplace/__pycache__/crawler_selenium.cpython-311.pyc View File


BIN
MarketPlaces/AnonymousMarketplace/__pycache__/parser.cpython-310.pyc View File


BIN
MarketPlaces/AnonymousMarketplace/__pycache__/parser.cpython-311.pyc View File


+ 1483
- 0
MarketPlaces/AnonymousMarketplace/geckodriver.log
File diff suppressed because it is too large
View File


BIN
MarketPlaces/Apocalypse/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
MarketPlaces/Apocalypse/__pycache__/crawler_selenium.cpython-311.pyc View File


BIN
MarketPlaces/Apocalypse/__pycache__/parser.cpython-310.pyc View File


BIN
MarketPlaces/Apocalypse/__pycache__/parser.cpython-311.pyc View File


+ 1483
- 0
MarketPlaces/Apocalypse/geckodriver.log
File diff suppressed because it is too large
View File


BIN
MarketPlaces/BlackPyramid/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
MarketPlaces/BlackPyramid/__pycache__/parser.cpython-310.pyc View File


+ 1483
- 0
MarketPlaces/BlackPyramid/geckodriver.log
File diff suppressed because it is too large
View File


BIN
MarketPlaces/CityMarket/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
MarketPlaces/CityMarket/__pycache__/crawler_selenium.cpython-311.pyc View File


BIN
MarketPlaces/CityMarket/__pycache__/parser.cpython-310.pyc View File


BIN
MarketPlaces/CityMarket/__pycache__/parser.cpython-311.pyc View File


+ 1483
- 0
MarketPlaces/CityMarket/geckodriver.log
File diff suppressed because it is too large
View File


BIN
MarketPlaces/Classifier/__pycache__/__init__.cpython-310.pyc View File


BIN
MarketPlaces/Classifier/__pycache__/__init__.cpython-311.pyc View File


BIN
MarketPlaces/Classifier/__pycache__/classify_product.cpython-310.pyc View File


BIN
MarketPlaces/Classifier/__pycache__/classify_product.cpython-311.pyc View File


BIN
MarketPlaces/Classifier/__pycache__/transformer.cpython-310.pyc View File


BIN
MarketPlaces/Classifier/__pycache__/transformer.cpython-311.pyc View File


+ 1483
- 0
MarketPlaces/CypherMarketplace/geckodriver.log
File diff suppressed because it is too large
View File


BIN
MarketPlaces/DB_Connection/__pycache__/__init__.cpython-310.pyc View File


BIN
MarketPlaces/DB_Connection/__pycache__/__init__.cpython-311.pyc View File


BIN
MarketPlaces/DB_Connection/__pycache__/db_connection.cpython-310.pyc View File


BIN
MarketPlaces/DB_Connection/__pycache__/db_connection.cpython-311.pyc View File


BIN
MarketPlaces/DarkFox/__pycache__/crawler_selenium.cpython-310.pyc View File


BIN
MarketPlaces/DarkFox/__pycache__/crawler_selenium.cpython-311.pyc View File


BIN
MarketPlaces/DarkFox/__pycache__/parser.cpython-310.pyc View File


Some files were not shown because too many files changed in this diff

Write Preview
Loading…
Cancel
Save